-- Logs begin at Sun 2022-08-07 11:32:44 UTC, end at Sun 2022-08-07 12:44:22 UTC. -- Aug 07 12:02:28.931818 dv-cinder-849846-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 07 12:02:28.941262 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 7 12:02:28 2022] *** Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: nodename: dv-cinder-849846-8-iscsi Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: machine: x86_64 Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: clock source: unix Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: pcre jit disabled Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: detected number of CPU cores: 4 Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: current working directory: / Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: detected binary path: /usr/bin/uwsgi-core Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: your processes number limit is 47680 Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: your memory page size is 4096 bytes Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: detected max file descriptor number: 2048 Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: lock engine: pthread robust mutexes Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: thunder lock: enabled Aug 07 12:02:28.943055 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 07 12:02:28.944544 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 07 12:02:28.983456 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: Python main interpreter initialized at 0x562b68cf69c0 Aug 07 12:02:28.983456 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: python threads support enabled Aug 07 12:02:28.983456 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: your server socket listen backlog is limited to 100 connections Aug 07 12:02:28.983456 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: your mercy for graceful operations on workers is 90 seconds Aug 07 12:02:28.983847 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: mapped 403077 bytes (393 KB) for 2 cores Aug 07 12:02:28.983925 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: *** Operational MODE: preforking *** Aug 07 12:02:28.984160 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: *** uWSGI is running in multiple interpreter mode *** Aug 07 12:02:28.984160 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: spawned uWSGI master process (pid: 80421) Aug 07 12:02:28.984372 dv-cinder-849846-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 07 12:02:28.989754 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: spawned uWSGI worker 1 (pid: 80432, cores: 1) Aug 07 12:02:28.989754 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: spawned uWSGI worker 2 (pid: 80433, cores: 1) Aug 07 12:02:28.989754 dv-cinder-849846-8-iscsi devstack@keystone.service[80421]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 07 12:02:34.536484 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 12:02:34.536484 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: from cryptography.utils import int_from_bytes Aug 07 12:02:34.537291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 12:02:34.537291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: from cryptography.utils import int_from_bytes Aug 07 12:02:34.537511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 12:02:34.537511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: from cryptography.utils import int_from_bytes Aug 07 12:02:34.537959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 12:02:34.537959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: from cryptography.utils import int_from_bytes Aug 07 12:02:34.586568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 12:02:34.586759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 12:02:34.586953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] command line args: [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 12:02:34.587075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 12:02:34.587250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 12:02:34.587337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 12:02:34.587435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ================================================================================ {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 12:02:34.587520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] command line args: [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 12:02:34.587694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] admin_token = **** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.587773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 12:02:34.587890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ================================================================================ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 12:02:34.587975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] config_dir = [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] admin_token = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] config_source = [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] config_dir = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588491 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] debug = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] config_source = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.588957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] debug = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.589087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.589148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.589233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.589385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.589497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.589605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.589709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.589806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.590027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.590620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] list_limit = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.590726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] log_config_append = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.590829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.590926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_config_append = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] log_dir = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_dir = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] log_file = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_file = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.591998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.592956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.593079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.593208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.593294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.593448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.593567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.593686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.593778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.593924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.594053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.594160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.594245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.594411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.594502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.594661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.594744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.594939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.595199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.595390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] publish_errors = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.595469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.595628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.595797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] publish_errors = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.595922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.595999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596496 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] standard_threads = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.596964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] standard_threads = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] use_journal = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] use_json = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_journal = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] use_stderr = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.597995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_json = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.598095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] use_syslog = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.598180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_stderr = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.598278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_syslog = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.598355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.598555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 12:02:34.598634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.598807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.598892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.599077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.599159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.599330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.599410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.599573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.599657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.599814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.599893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.600060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.600149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.600299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.600375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.600543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.600624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.600796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.600884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.601049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.601190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.601300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.601507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.601586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.601777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.601856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.602019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.602102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.602273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.602407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.602527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.602711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.602789 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.603033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.603111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.603284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.603374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.603536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.603657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.603775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.603983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.604067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.604297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.604380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.604559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.604643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.604815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.605013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.605168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.605278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.605362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.605514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.606283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.606379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.606536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.606715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.606863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.607049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.607163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.607365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.607454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.607651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.607728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.607908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.608013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.608153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.608355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.608434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.608633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.608709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.608875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.608953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.609116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.609262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.609336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.609578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.609644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.609815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.609891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.610052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.610207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.610277 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.610450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.610545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.610694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.610774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.610937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.611034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.611210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.611305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.611447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.611795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.611948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.612988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.613143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.613232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.613375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.613452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.613612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.613686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.613847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.613926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.614095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.614172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.614331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.614409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.614570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.614648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.614806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.614884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.615070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.615150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.615312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.615401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.615547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.615624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.615777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.615855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.616991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.617071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.617231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.617310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.617466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.617555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.617704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.617782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.617942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.618985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.619163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.619244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.619399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.619476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.619635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.619725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.619874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.619954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.620118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.620196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.620356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.620434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.620600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.620687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.620838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.620917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.621077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.621158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.621913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.622004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.622241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.622326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.622486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.622613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.622754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.622857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.623999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.624078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.624239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.624319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.624478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.624557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.624714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.624793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.624948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625028 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625181 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.external = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.625971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.626129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.626208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.626380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.626460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.626618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.626697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.password = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.626855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.token = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.626939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.627139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.627252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.627380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.627461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.627623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] auth.external = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.627708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.627857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.627945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.628107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.628187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.628362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.628441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.628599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] auth.password = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.628677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.628835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] auth.token = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.628916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.629078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.629157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.629315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.629402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.629564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.629645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.629799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.629878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.630046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.630132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.630286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.630372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.630517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.630595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.630755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] credential.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.630843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.631056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.631143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.631282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.631364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.631537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.631625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.631778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.631858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.632026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.632106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.632263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.632350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.632531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.632611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.632767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.632846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.633989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.634069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.634231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.634311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.634491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.634580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.634728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] federation.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.634806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.634968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.635076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.635238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.635326 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.635488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.635567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.635723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.635803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.635973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.636060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.636213 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.636292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.636451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.636528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.636690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.636778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.637088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.637185 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.637333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.637413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.637583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.637714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.637825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.637972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.638079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.638232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.638332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.638472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.638560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.638708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.638788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.638945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.639044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.639222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.639304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.639476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.639556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.639731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.639818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.639985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.640064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.640222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.640301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.640456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.640544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.640690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.640770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.640939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.641995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.642150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.642230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.642396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.642477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.642635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.642722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.642869 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.642956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.643134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.643217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.643369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.643448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.643606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.643676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.643848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.644886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.644970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.644970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.644970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.644970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.645954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.646087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.646168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.646336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.646418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.646580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.646669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.646825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.646906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.647102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.647185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.647345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.647439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.647585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.647666 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.647826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.647905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.648076 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.648163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.648312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.648393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.648545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.648624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.648785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.648871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649028 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.649977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.650057 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.650220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.650308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.650469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.650547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.650701 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.650779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.650948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.651080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.651209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.651290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.651451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.651531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.651686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.651773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.651920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.652000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.652154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.652232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.652387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.652467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.652660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.652780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.652894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.653248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.653352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.653429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.653501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.653621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.653702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.653868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.653957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.654116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.654196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.654348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.654428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.654598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.654687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.654838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.654919 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.655099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.655183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.655340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.655433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.655578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.655659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.655821 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.655900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.656069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.656156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.656303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.656382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.656554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.656633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.656785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.656943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657805 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.657965 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.658043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.658205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.658293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.658439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.658517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.658678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.658758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.658920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] resource.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.659990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.660163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.660243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.660399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.660486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.660641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.660719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.660877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.660956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] role.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.661118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.661206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] role.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.661351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.661429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] role.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.661598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] role.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.661679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.661831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.661920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.662079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.662159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.662328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.662409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.662578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.662665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.662825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.662903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.663095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.663178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.663346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.663437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.663579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.663658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.663829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.663909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.664069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.664156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.664319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.664399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.664569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.664648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.664816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.664970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.665051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.665211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.665450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.665530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.665691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.665772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.665943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.666023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.666193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.666271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.666433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.666550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.666804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.666884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.667061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.667142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.667323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.667405 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.667563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.667641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.667810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.667890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.668047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.668660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.668822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.672972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.673196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.673196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.674874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675211 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675313 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.675937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] token.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.676993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.677947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.connection = **** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678711 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.678956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.679970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.680979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.681049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.681127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.681127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 12:02:34.681258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.681258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.681258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 12:02:34.681258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 12:02:34.708576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.708809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.708892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.709039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.709125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.709272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.719305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.719568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.748167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.748393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.748585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.748820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.800062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.801084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.850536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.850768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.852523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.852763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.915984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.916208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.916414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.916521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.916639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.916721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.916856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.916940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.917073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.917166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.917291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.917509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.940601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.940835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.940961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941201 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.941980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.942160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.942380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.953036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.953595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.966485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.966713 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.966846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.966930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.967995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.968211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 12:02:34.985202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.985415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 12:02:34.985768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.985929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.986015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 12:02:34.986146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 12:02:34.986229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.986731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.986820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.986942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 12:02:34.987047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.987254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.987338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.987522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.987814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.988075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.998977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.999282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.999558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.999845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:34.999937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.000169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.000451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.000742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.006812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.007777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.009965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.010256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.010543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.010827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.010921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.011144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.011229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.011424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.011519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.011707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.011815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.012114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.012198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.012385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.012467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.012663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.012747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.012945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.013027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.013222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.013502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.013777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.036806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.037453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.038334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.038607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.047188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.047188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.047366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.047366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.047366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.047580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.047580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.047580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.051029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.051287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.051952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.052245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.055326 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.055620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.055915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.056207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.056382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.056494 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.056679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.056762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.056977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.057069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.057269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.057352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.057564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.057858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.058167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.058445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.073034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.073321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.075476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.075828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.077998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.080016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.081194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.081497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.083216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.083521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.085913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.087969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.089137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.091277 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.092451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.092757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.093051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.093352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.093636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.093933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.094456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.094765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.095083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.095384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.095670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.095964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.110530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.110833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.111129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.111399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.113095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.113403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.113683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.113961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.118539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.120150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.120432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.120929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.123173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.123461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.124105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.124395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.124700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.124996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.125279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.126290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.126582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.126889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.127221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.127507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.136745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.137676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.137965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.138253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.138538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.138820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.139128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.139734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.140026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.140315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.140602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.140895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.141181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.141467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.156547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.159025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.159550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.162011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.162494 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.164103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.164999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.165603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.165891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.166186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.166595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.168171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.168472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.168771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.172828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.174441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.175296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.176921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.177394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.177699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.177995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.178280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.179891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.180192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.180487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.180778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.186639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.186932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.187265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.189010 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.189309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.189600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.194153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.194444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.194730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.195030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.195314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.195593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.195875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.196161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.196534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 12:02:35.196617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.196821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.196910 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.197102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.197183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.197372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.197452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.197654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.197735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.197935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.198016 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.198229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.198516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.198902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.199217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.200119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.200357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.200650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.200922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 12:02:35.233486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562b68cf69c0 pid: 80432 (default app) Aug 07 12:02:35.235143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562b68cf69c0 pid: 80433 (default app) Aug 07 12:02:35.237018 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d0c48-7458-41e6-87e2-9a076e6d2400 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:35.237308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d0c48-7458-41e6-87e2-9a076e6d2400 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:35.237584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d0c48-7458-41e6-87e2-9a076e6d2400 None None] PATH_INFO: `/v3` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:35.241915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Sun Aug 7 12:02:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:02:43.748452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93d515-8bc5-4bb5-b568-df8153950735 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:43.749011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93d515-8bc5-4bb5-b568-df8153950735 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:43.749011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93d515-8bc5-4bb5-b568-df8153950735 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:43.753503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:43.758922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:43.759310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:43.759499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:43.829522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: INFO dbcounter [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] Registered counter for database keystone Aug 07 12:02:43.848909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG oslo_db.sqlalchemy.engines [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80432) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 12:02:43.855096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writer thread running {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 12:02:44.034145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG passlib.handlers.bcrypt [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80432) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 07 12:02:44.034487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG passlib.handlers.bcrypt [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80432) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 07 12:02:44.088934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:44.091247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:44.224883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 07 12:02:44.360701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93d515-8bc5-4bb5-b568-df8153950735 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:44.429969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: INFO dbcounter [None req-3e93d515-8bc5-4bb5-b568-df8153950735 None None] Registered counter for database keystone Aug 07 12:02:44.451744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG oslo_db.sqlalchemy.engines [None req-3e93d515-8bc5-4bb5-b568-df8153950735 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80433) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 12:02:44.457944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writer thread running {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 12:02:44.576929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93d515-8bc5-4bb5-b568-df8153950735 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:44.581217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b88453-77ed-45ee-a49e-5b65e4935e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:44.581839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88453-77ed-45ee-a49e-5b65e4935e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:44.582121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88453-77ed-45ee-a49e-5b65e4935e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:44.582403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88453-77ed-45ee-a49e-5b65e4935e28 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:44.590968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b88453-77ed-45ee-a49e-5b65e4935e28 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:44.642814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d8b88453-77ed-45ee-a49e-5b65e4935e28 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 07 12:02:44.643187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b88453-77ed-45ee-a49e-5b65e4935e28 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:44.653415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Sun Aug 7 12:02:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:02:44.659216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:44.674447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c565d17-9006-41a2-9560-363b044b6851 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:44.678881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4a0fd-2d94-4dfa-ba34-793701627c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:44.680029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4a0fd-2d94-4dfa-ba34-793701627c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:44.680316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4a0fd-2d94-4dfa-ba34-793701627c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:44.680595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4a0fd-2d94-4dfa-ba34-793701627c62 admin admin] PATH_INFO: `/v3/projects/7cc097a340cd405e932b3797d5864d76` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:44.682033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4a0fd-2d94-4dfa-ba34-793701627c62 admin admin] RBAC: Authorizing `identity:get_project(project_id=7cc097a340cd405e932b3797d5864d76)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:44.733458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eae4a0fd-2d94-4dfa-ba34-793701627c62 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 07 12:02:44.733805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4a0fd-2d94-4dfa-ba34-793701627c62 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:44.738701 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Sun Aug 7 12:02:44 2022] GET /identity/v3/projects/7cc097a340cd405e932b3797d5864d76 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:46.694220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bdb87-3b08-4d24-b124-c68994268cc2 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:46.694737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bdb87-3b08-4d24-b124-c68994268cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:46.694822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bdb87-3b08-4d24-b124-c68994268cc2 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:46.698889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:02:46.704561 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aafb7a-80fe-4483-bc10-d314d409818e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:46.704932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aafb7a-80fe-4483-bc10-d314d409818e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:46.705109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aafb7a-80fe-4483-bc10-d314d409818e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:46.726308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-77aafb7a-80fe-4483-bc10-d314d409818e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:46.727851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-77aafb7a-80fe-4483-bc10-d314d409818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:46.736176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 07 12:02:46.759484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bdb87-3b08-4d24-b124-c68994268cc2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:46.771650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bdb87-3b08-4d24-b124-c68994268cc2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:46.775853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37680b-5ed1-4798-957f-2147af963fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:46.776444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37680b-5ed1-4798-957f-2147af963fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:46.776725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37680b-5ed1-4798-957f-2147af963fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:46.776996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37680b-5ed1-4798-957f-2147af963fd2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:46.785287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37680b-5ed1-4798-957f-2147af963fd2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:46.787062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37680b-5ed1-4798-957f-2147af963fd2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:46.796403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Sun Aug 7 12:02:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:02:46.801605 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aafb7a-80fe-4483-bc10-d314d409818e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:46.810743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aafb7a-80fe-4483-bc10-d314d409818e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:46.815116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84dd7f-a95b-4b0d-b3a5-faf1fc253c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:46.815739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84dd7f-a95b-4b0d-b3a5-faf1fc253c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:46.816020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84dd7f-a95b-4b0d-b3a5-faf1fc253c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:46.816294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84dd7f-a95b-4b0d-b3a5-faf1fc253c60 admin admin] PATH_INFO: `/v3/users/6e390f818f21433d9d55c5e4c3ba79e5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:46.817613 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84dd7f-a95b-4b0d-b3a5-faf1fc253c60 admin admin] RBAC: Authorizing `identity:get_user(user_id=6e390f818f21433d9d55c5e4c3ba79e5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:46.819471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84dd7f-a95b-4b0d-b3a5-faf1fc253c60 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:46.820963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Sun Aug 7 12:02:46 2022] GET /identity/v3/users/6e390f818f21433d9d55c5e4c3ba79e5 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:02:48.849840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ba471-fd8a-4231-94c6-298ff798363d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:48.850451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ba471-fd8a-4231-94c6-298ff798363d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:48.850451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ba471-fd8a-4231-94c6-298ff798363d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:48.851747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:48.857463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5fa24-2265-4d69-bfcf-770466c9d388 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:48.857904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5fa24-2265-4d69-bfcf-770466c9d388 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:48.858008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5fa24-2265-4d69-bfcf-770466c9d388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:48.876255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d6a5fa24-2265-4d69-bfcf-770466c9d388 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:48.877421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d6a5fa24-2265-4d69-bfcf-770466c9d388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:48.884643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339ffc-ac93-4e39-af76-d29d14993a54 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:48.885368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339ffc-ac93-4e39-af76-d29d14993a54 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:48.885658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339ffc-ac93-4e39-af76-d29d14993a54 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:48.887345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:02:48.892316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:48.892605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:48.892886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:48.935838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG passlib.handlers.bcrypt [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80433) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 07 12:02:48.936267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG passlib.handlers.bcrypt [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80433) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 07 12:02:48.955316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:02:48.977968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5fa24-2265-4d69-bfcf-770466c9d388 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:48.982500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:48.984127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:49.001221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:49.084553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5fa24-2265-4d69-bfcf-770466c9d388 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:49.084932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:02:49.106688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:49.123772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf10474-3fc5-43d0-b1dc-f54f1d6602ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:49.124424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf10474-3fc5-43d0-b1dc-f54f1d6602ad None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:49.124703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf10474-3fc5-43d0-b1dc-f54f1d6602ad None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:49.124979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf10474-3fc5-43d0-b1dc-f54f1d6602ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:49.137158 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf10474-3fc5-43d0-b1dc-f54f1d6602ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-fbf10474-3fc5-43d0-b1dc-f54f1d6602ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:02:49.145574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:02:49.146630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:02:49.147694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:02:49.148509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:02:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:02:49.152532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf10474-3fc5-43d0-b1dc-f54f1d6602ad None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:49.210721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a9923-de63-409a-bd6f-1c955816acdf None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:49.213000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:49.235100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:49.248100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba6816-beb5-4f1f-9398-80cb3ba3ce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:49.248982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6816-beb5-4f1f-9398-80cb3ba3ce78 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:49.249266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6816-beb5-4f1f-9398-80cb3ba3ce78 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:49.249542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6816-beb5-4f1f-9398-80cb3ba3ce78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:49.253150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf10474-3fc5-43d0-b1dc-f54f1d6602ad None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:49.286860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:49.287506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:49.287796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:49.288071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:49.288740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:02:49.289043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:49.290443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:49.302848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:02:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:49.309133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:49.318194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba6816-beb5-4f1f-9398-80cb3ba3ce78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:49.319756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba6816-beb5-4f1f-9398-80cb3ba3ce78 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:49.408824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a2751-09bb-41c8-bbed-b440f4e2d6b0 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:49.416325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 8/16] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:02:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:02:49.422336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba6816-beb5-4f1f-9398-80cb3ba3ce78 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:49.425550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:49.529100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba6816-beb5-4f1f-9398-80cb3ba3ce78 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:49.549634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f0be-e2e0-4b00-ba6d-72ec7ec8ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:49.550422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f0be-e2e0-4b00-ba6d-72ec7ec8ee7d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:49.550709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f0be-e2e0-4b00-ba6d-72ec7ec8ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:49.551024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f0be-e2e0-4b00-ba6d-72ec7ec8ee7d None admin] PATH_INFO: `/v3/users/6e390f818f21433d9d55c5e4c3ba79e5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:49.552175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585f0be-e2e0-4b00-ba6d-72ec7ec8ee7d None admin] RBAC: Authorizing `identity:get_user(user_id=6e390f818f21433d9d55c5e4c3ba79e5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:49.553630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585f0be-e2e0-4b00-ba6d-72ec7ec8ee7d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:49.555220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:02:49 2022] GET /identity/v3/users/6e390f818f21433d9d55c5e4c3ba79e5 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:49.560563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f0be-e2e0-4b00-ba6d-72ec7ec8ee7d None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:49.562499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:49.563369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:49.563644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:49.563915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:49.576092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:49.582894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:49.585942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:02:49.603104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:02:49.604094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:02:49.605148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:02:49.605148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:02:49.605148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:02:49.605148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 9/18] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:02:49.610037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:49.664921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f0be-e2e0-4b00-ba6d-72ec7ec8ee7d None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:49.667217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:49.698668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e77ae-eafb-40a4-b0e5-1a0791eacd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:49.699450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e77ae-eafb-40a4-b0e5-1a0791eacd54 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:49.699839 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e77ae-eafb-40a4-b0e5-1a0791eacd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:49.700111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e77ae-eafb-40a4-b0e5-1a0791eacd54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:49.701234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e77ae-eafb-40a4-b0e5-1a0791eacd54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:49.702684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e77ae-eafb-40a4-b0e5-1a0791eacd54 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:49.704238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 10/19] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:02:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:49.710005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e77ae-eafb-40a4-b0e5-1a0791eacd54 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:49.711041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a437c21-51c8-4c40-a111-8ccbe231fcd8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:49.745288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2df20e-8cb3-4cce-877b-723989b4f799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:49.746397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df20e-8cb3-4cce-877b-723989b4f799 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:49.746677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df20e-8cb3-4cce-877b-723989b4f799 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:49.746961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df20e-8cb3-4cce-877b-723989b4f799 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:49.747638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2df20e-8cb3-4cce-877b-723989b4f799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:02:49.747939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2df20e-8cb3-4cce-877b-723989b4f799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:49.749315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2df20e-8cb3-4cce-877b-723989b4f799 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:49.766981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 10/20] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:49.796678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:49.816601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e77ae-eafb-40a4-b0e5-1a0791eacd54 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:49.853700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ce795-5376-420b-9cf3-0e63d980b48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:49.854320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ce795-5376-420b-9cf3-0e63d980b48e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:49.854601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ce795-5376-420b-9cf3-0e63d980b48e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:49.854879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ce795-5376-420b-9cf3-0e63d980b48e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:49.856633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ce795-5376-420b-9cf3-0e63d980b48e None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.domain.id=default, user.id=6e390f818f21433d9d55c5e4c3ba79e5) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:02:49.857010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ce795-5376-420b-9cf3-0e63d980b48e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:49.858489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ce795-5376-420b-9cf3-0e63d980b48e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:49.866180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 11/21] 10.222.0.30 () {56 vars in 1322 bytes} [Sun Aug 7 12:02:49 2022] GET /identity/v3/role_assignments?user.id=6e390f818f21433d9d55c5e4c3ba79e5&scope.domain.id=default&role.id=f72d3388f13149b483355331380f040b => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:52.059737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d525283-536c-4b4e-919b-a3cfe37a452f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:52.060251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d525283-536c-4b4e-919b-a3cfe37a452f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:52.060358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d525283-536c-4b4e-919b-a3cfe37a452f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:52.061676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 11/22] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:52.066808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45216b-0917-49bd-891a-5c083962e0b5 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:52.067235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45216b-0917-49bd-891a-5c083962e0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:52.067408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45216b-0917-49bd-891a-5c083962e0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:52.091152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3c45216b-0917-49bd-891a-5c083962e0b5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:52.092363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3c45216b-0917-49bd-891a-5c083962e0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:52.194595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 12/23] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:02:52.228342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d525283-536c-4b4e-919b-a3cfe37a452f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:52.508283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d525283-536c-4b4e-919b-a3cfe37a452f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:52.518121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add7e91-0bb5-4239-bff7-8020709daf30 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:52.518436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add7e91-0bb5-4239-bff7-8020709daf30 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:52.518689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add7e91-0bb5-4239-bff7-8020709daf30 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:52.520328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 13/24] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:52.530930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9050ef-c742-4650-9f22-1db87ab01fa3 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:52.531247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9050ef-c742-4650-9f22-1db87ab01fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:52.531531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9050ef-c742-4650-9f22-1db87ab01fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:52.563781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:52.572021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bd9050ef-c742-4650-9f22-1db87ab01fa3 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:52.573443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bd9050ef-c742-4650-9f22-1db87ab01fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:52.597785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf598b-0aad-42a3-87e6-dea76b3c5ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:52.598431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf598b-0aad-42a3-87e6-dea76b3c5ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:52.599555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf598b-0aad-42a3-87e6-dea76b3c5ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:52.599555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf598b-0aad-42a3-87e6-dea76b3c5ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:52.631026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf598b-0aad-42a3-87e6-dea76b3c5ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:52.631026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf598b-0aad-42a3-87e6-dea76b3c5ce1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c9cf598b-0aad-42a3-87e6-dea76b3c5ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:02:52.675036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:02:52.675956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:02:52.676780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:02:52.676780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:02:52.676780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:02:52.688714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 12/25] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:02:52.690224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca11c9d-2ac0-4f79-9b4a-58b5a3dd0f99 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:52.690641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca11c9d-2ac0-4f79-9b4a-58b5a3dd0f99 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:52.699144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca11c9d-2ac0-4f79-9b4a-58b5a3dd0f99 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:52.701104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 13/26] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:52 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:52.702043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca11c9d-2ac0-4f79-9b4a-58b5a3dd0f99 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:52.753069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 14/27] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:02:52.758815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225430c1-066b-4149-8083-dea9fca6f064 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:52.759495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225430c1-066b-4149-8083-dea9fca6f064 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:52.759986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225430c1-066b-4149-8083-dea9fca6f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:52.767721 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:52.781286 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-225430c1-066b-4149-8083-dea9fca6f064 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:52.782818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-225430c1-066b-4149-8083-dea9fca6f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:52.880589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca11c9d-2ac0-4f79-9b4a-58b5a3dd0f99 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:52.900710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 15/28] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:02:52.901418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225430c1-066b-4149-8083-dea9fca6f064 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:53.031662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c13c6-c83a-43b1-b9ea-4091aac1cdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:53.032262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c13c6-c83a-43b1-b9ea-4091aac1cdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.032572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c13c6-c83a-43b1-b9ea-4091aac1cdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.032846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c13c6-c83a-43b1-b9ea-4091aac1cdd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.033773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c13c6-c83a-43b1-b9ea-4091aac1cdd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:02:53.034082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c13c6-c83a-43b1-b9ea-4091aac1cdd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:53.035489 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c13c6-c83a-43b1-b9ea-4091aac1cdd3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:53.035983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225430c1-066b-4149-8083-dea9fca6f064 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:53.049344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 14/29] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:02:53.050955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66adc1-7d82-4159-8a34-69b1c1257bb1 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.051498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66adc1-7d82-4159-8a34-69b1c1257bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.051986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66adc1-7d82-4159-8a34-69b1c1257bb1 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.053761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 15/30] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:53.054702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66adc1-7d82-4159-8a34-69b1c1257bb1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:53.088729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81907190-5e1f-4124-b65e-ba6f585d553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:53.090641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81907190-5e1f-4124-b65e-ba6f585d553e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.091227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81907190-5e1f-4124-b65e-ba6f585d553e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.091416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:53.092527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81907190-5e1f-4124-b65e-ba6f585d553e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.094928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:53.104786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81907190-5e1f-4124-b65e-ba6f585d553e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-81907190-5e1f-4124-b65e-ba6f585d553e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:02:53.111734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:02:53.112825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:02:53.113658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:02:53.116886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 16/31] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:02:53.118385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81907190-5e1f-4124-b65e-ba6f585d553e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:53.194122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66adc1-7d82-4159-8a34-69b1c1257bb1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:53.232870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81907190-5e1f-4124-b65e-ba6f585d553e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:53.241155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fa174-e579-48be-bcb1-d004c3637536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:53.241768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fa174-e579-48be-bcb1-d004c3637536 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.242048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fa174-e579-48be-bcb1-d004c3637536 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.242819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fa174-e579-48be-bcb1-d004c3637536 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.243542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fa174-e579-48be-bcb1-d004c3637536 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:53.245363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fa174-e579-48be-bcb1-d004c3637536 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:53.278687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a088457-76d2-4cc8-8b97-cc3ca8cf1476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:53.279366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a088457-76d2-4cc8-8b97-cc3ca8cf1476 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.279653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a088457-76d2-4cc8-8b97-cc3ca8cf1476 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.279926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a088457-76d2-4cc8-8b97-cc3ca8cf1476 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.280571 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a088457-76d2-4cc8-8b97-cc3ca8cf1476 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:53.282149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a088457-76d2-4cc8-8b97-cc3ca8cf1476 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:53.440248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 16/32] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 12:02:53 2022] POST /identity/v3/roles => generated 219 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:02:53.442160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211aba1-d2a7-4532-9e34-405a6d03561a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.442660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211aba1-d2a7-4532-9e34-405a6d03561a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.443182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211aba1-d2a7-4532-9e34-405a6d03561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.467019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1211aba1-d2a7-4532-9e34-405a6d03561a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:53.468662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1211aba1-d2a7-4532-9e34-405a6d03561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:53.473066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 17/33] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:02:53 2022] POST /identity/v3/projects => generated 301 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:02:53.474653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a088457-76d2-4cc8-8b97-cc3ca8cf1476 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:53.582795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:53.639479 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 17/34] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:02:53.641432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7364-cf9c-4d68-9f47-b92e405ecf23 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.641994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7364-cf9c-4d68-9f47-b92e405ecf23 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.642476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7364-cf9c-4d68-9f47-b92e405ecf23 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.644147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 18/35] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:53.653879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cb6d2-2dbc-4cf1-90ac-2ac5b0b23cf5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.654172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cb6d2-2dbc-4cf1-90ac-2ac5b0b23cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.654435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cb6d2-2dbc-4cf1-90ac-2ac5b0b23cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.685817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-017cb6d2-2dbc-4cf1-90ac-2ac5b0b23cf5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:53.688245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-017cb6d2-2dbc-4cf1-90ac-2ac5b0b23cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:53.696468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a088457-76d2-4cc8-8b97-cc3ca8cf1476 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:53.704283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:53.749167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:53.750260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.750754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.751263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.752215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:02:53.752788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:53.754281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:53.768888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 18/36] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:02:53.769781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:53.776078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 19/37] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:02:53.777850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cb6d2-2dbc-4cf1-90ac-2ac5b0b23cf5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:53.876785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980165cf-508a-4583-ba99-f5376ef3ac30 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:53.886771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:53.886932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cb6d2-2dbc-4cf1-90ac-2ac5b0b23cf5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:53.911107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77bdcd-22ef-4e56-a8d2-7302d2ebba26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:53.911711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77bdcd-22ef-4e56-a8d2-7302d2ebba26 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.911989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77bdcd-22ef-4e56-a8d2-7302d2ebba26 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.912258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77bdcd-22ef-4e56-a8d2-7302d2ebba26 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.912976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77bdcd-22ef-4e56-a8d2-7302d2ebba26 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:53.914361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77bdcd-22ef-4e56-a8d2-7302d2ebba26 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:53.921095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f70ec-1878-4d18-95f4-dfd66ce648a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:53.921720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f70ec-1878-4d18-95f4-dfd66ce648a8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.922001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f70ec-1878-4d18-95f4-dfd66ce648a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.922275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f70ec-1878-4d18-95f4-dfd66ce648a8 None admin] PATH_INFO: `/v3/users/6e390f818f21433d9d55c5e4c3ba79e5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:53.923793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f70ec-1878-4d18-95f4-dfd66ce648a8 None admin] RBAC: Authorizing `identity:get_user(user_id=6e390f818f21433d9d55c5e4c3ba79e5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:53.925342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f70ec-1878-4d18-95f4-dfd66ce648a8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:53.926898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 20/38] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:02:53 2022] GET /identity/v3/users/6e390f818f21433d9d55c5e4c3ba79e5 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:02:53.927741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f70ec-1878-4d18-95f4-dfd66ce648a8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:53.945908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:53.993815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 19/39] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 12:02:53 2022] POST /identity/v3/roles => generated 225 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:02:53.995183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d89d0e-ef95-4b19-8fb4-bc2682fbb371 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:53.995466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d89d0e-ef95-4b19-8fb4-bc2682fbb371 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:53.995812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d89d0e-ef95-4b19-8fb4-bc2682fbb371 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:54.001094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 20/40] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:54.001094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d89d0e-ef95-4b19-8fb4-bc2682fbb371 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:54.038879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f70ec-1878-4d18-95f4-dfd66ce648a8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:54.077161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c157a-991e-438f-b572-12d1de3f8a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:54.077792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c157a-991e-438f-b572-12d1de3f8a57 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:54.078076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c157a-991e-438f-b572-12d1de3f8a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:54.078364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c157a-991e-438f-b572-12d1de3f8a57 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:54.079083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c157a-991e-438f-b572-12d1de3f8a57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:54.080497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c157a-991e-438f-b572-12d1de3f8a57 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:54.122212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d89d0e-ef95-4b19-8fb4-bc2682fbb371 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:54.158191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d0ea1-2dff-43a2-835a-6d42b7b644d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:54.158751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d0ea1-2dff-43a2-835a-6d42b7b644d8 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:54.159064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d0ea1-2dff-43a2-835a-6d42b7b644d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:54.159343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d0ea1-2dff-43a2-835a-6d42b7b644d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:54.160566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d0ea1-2dff-43a2-835a-6d42b7b644d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:54.162081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d0ea1-2dff-43a2-835a-6d42b7b644d8 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:54.163748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 21/41] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:02:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:02:54.169336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f36f-3a91-40d5-9744-68d5f6144a98 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:54.169336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f36f-3a91-40d5-9744-68d5f6144a98 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:54.169336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f36f-3a91-40d5-9744-68d5f6144a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:54.182766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 21/42] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 12:02:53 2022] POST /identity/v3/roles => generated 223 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:02:54.184265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a6b4f36f-3a91-40d5-9744-68d5f6144a98 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:54.185953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a6b4f36f-3a91-40d5-9744-68d5f6144a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:54.186870 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c157a-991e-438f-b572-12d1de3f8a57 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:54.230444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:54.278449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 22/43] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:02:54.284917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:54.302743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f36f-3a91-40d5-9744-68d5f6144a98 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:54.310767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c157a-991e-438f-b572-12d1de3f8a57 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:54.352174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459447f-9a21-4c55-ac3b-eea936f9251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:54.352847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459447f-9a21-4c55-ac3b-eea936f9251d None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:54.353128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459447f-9a21-4c55-ac3b-eea936f9251d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:54.353407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459447f-9a21-4c55-ac3b-eea936f9251d None admin] PATH_INFO: `/v3/domains/default/users/6e390f818f21433d9d55c5e4c3ba79e5/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:54.355619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459447f-9a21-4c55-ac3b-eea936f9251d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6e390f818f21433d9d55c5e4c3ba79e5, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:54.357101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459447f-9a21-4c55-ac3b-eea936f9251d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:54.416348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f36f-3a91-40d5-9744-68d5f6144a98 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:54.426414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 22/44] 10.222.0.30 () {58 vars in 1361 bytes} [Sun Aug 7 12:02:54 2022] PUT /identity/v3/domains/default/users/6e390f818f21433d9d55c5e4c3ba79e5/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:02:54.452802 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0644c1-3eef-436e-8b87-02d9c0d6542a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:54.453464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0644c1-3eef-436e-8b87-02d9c0d6542a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:54.453746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0644c1-3eef-436e-8b87-02d9c0d6542a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:54.454018 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0644c1-3eef-436e-8b87-02d9c0d6542a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:54.455250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0644c1-3eef-436e-8b87-02d9c0d6542a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:54.456758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0644c1-3eef-436e-8b87-02d9c0d6542a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:54.458394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 23/45] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:02:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:54.464468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459447f-9a21-4c55-ac3b-eea936f9251d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:54.572604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459447f-9a21-4c55-ac3b-eea936f9251d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:54.607272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acce8b5-2216-45d5-936b-ff4c56cdb199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:54.607894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acce8b5-2216-45d5-936b-ff4c56cdb199 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:54.608176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acce8b5-2216-45d5-936b-ff4c56cdb199 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:54.608451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acce8b5-2216-45d5-936b-ff4c56cdb199 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:54.609101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acce8b5-2216-45d5-936b-ff4c56cdb199 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:54.610554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acce8b5-2216-45d5-936b-ff4c56cdb199 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:54.616516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:54.696680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 23/46] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:02:54 2022] POST /identity/v3/projects => generated 312 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:02:56.405421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2cc38-2bc2-41bd-ade3-17b8022eb63e None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:56.405942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2cc38-2bc2-41bd-ade3-17b8022eb63e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:56.406024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2cc38-2bc2-41bd-ade3-17b8022eb63e None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:56.407336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 24/47] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:56.412596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f960e-e180-4532-85cf-c5a5bfee664b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:56.412978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f960e-e180-4532-85cf-c5a5bfee664b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:56.413159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f960e-e180-4532-85cf-c5a5bfee664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:56.431660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5d1f960e-e180-4532-85cf-c5a5bfee664b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:56.432851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5d1f960e-e180-4532-85cf-c5a5bfee664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:56.498879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 24/48] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:02:56.521650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2cc38-2bc2-41bd-ade3-17b8022eb63e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:56.560538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:56.626773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2cc38-2bc2-41bd-ade3-17b8022eb63e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:56.659276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8bdab-abb7-4e71-9d7a-1ede07d81671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:56.659831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8bdab-abb7-4e71-9d7a-1ede07d81671 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:56.660115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8bdab-abb7-4e71-9d7a-1ede07d81671 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:56.660391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8bdab-abb7-4e71-9d7a-1ede07d81671 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:56.670713 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8bdab-abb7-4e71-9d7a-1ede07d81671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-a2b8bdab-abb7-4e71-9d7a-1ede07d81671 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:02:56.675501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:02:56.676441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:02:56.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 25/49] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:02:56.682594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f960e-e180-4532-85cf-c5a5bfee664b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:56.706600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8394e0-6957-4739-b00a-7ceac6f5400f None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:56.706893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8394e0-6957-4739-b00a-7ceac6f5400f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:56.707212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8394e0-6957-4739-b00a-7ceac6f5400f None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:56.708548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 26/50] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:56.713219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb7c8-2028-41de-acdf-ab0cee538e34 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:56.713510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb7c8-2028-41de-acdf-ab0cee538e34 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:56.713813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb7c8-2028-41de-acdf-ab0cee538e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:56.732447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-609cb7c8-2028-41de-acdf-ab0cee538e34 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:56.733652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-609cb7c8-2028-41de-acdf-ab0cee538e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:56.782779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f960e-e180-4532-85cf-c5a5bfee664b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:56.800095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:56.800618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 27/51] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:02:56.816562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:56.817178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:56.817463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:56.817741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:56.818415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:02:56.818726 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:56.821394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:56.823262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb7c8-2028-41de-acdf-ab0cee538e34 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:56.833899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 25/52] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:02:56.839919 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:56.848118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:56.921260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb7c8-2028-41de-acdf-ab0cee538e34 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:56.938470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b049f-a1b0-4a44-abba-afd82657e734 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:56.954666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d9e60-9549-494d-9282-e8cecc739e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:56.955335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d9e60-9549-494d-9282-e8cecc739e33 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:56.955624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d9e60-9549-494d-9282-e8cecc739e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:56.955904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d9e60-9549-494d-9282-e8cecc739e33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:56.957030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d9e60-9549-494d-9282-e8cecc739e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:56.958502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d9e60-9549-494d-9282-e8cecc739e33 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:56.960087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 28/53] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:02:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:56.966270 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d9e60-9549-494d-9282-e8cecc739e33 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:56.971969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16adb94-f144-47aa-bb01-5137d115b58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:56.972593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16adb94-f144-47aa-bb01-5137d115b58d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:56.972875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16adb94-f144-47aa-bb01-5137d115b58d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:56.973148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16adb94-f144-47aa-bb01-5137d115b58d None admin] PATH_INFO: `/v3/users/6e390f818f21433d9d55c5e4c3ba79e5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:56.974274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16adb94-f144-47aa-bb01-5137d115b58d None admin] RBAC: Authorizing `identity:get_user(user_id=6e390f818f21433d9d55c5e4c3ba79e5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:56.975743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16adb94-f144-47aa-bb01-5137d115b58d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:56.977334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 26/54] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:02:56 2022] GET /identity/v3/users/6e390f818f21433d9d55c5e4c3ba79e5 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:56.983239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16adb94-f144-47aa-bb01-5137d115b58d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:57.030592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:57.055744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:57.064197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d9e60-9549-494d-9282-e8cecc739e33 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:57.082046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16adb94-f144-47aa-bb01-5137d115b58d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:57.098605 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537c047-e857-4fe4-8a88-674c70bad023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:57.099233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537c047-e857-4fe4-8a88-674c70bad023 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:57.099518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537c047-e857-4fe4-8a88-674c70bad023 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:57.099787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537c047-e857-4fe4-8a88-674c70bad023 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:57.100421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537c047-e857-4fe4-8a88-674c70bad023 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:57.101853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537c047-e857-4fe4-8a88-674c70bad023 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:57.119657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803346e-9ba4-444e-b222-47c512a710a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:57.120305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803346e-9ba4-444e-b222-47c512a710a2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:57.120687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803346e-9ba4-444e-b222-47c512a710a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:57.121062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803346e-9ba4-444e-b222-47c512a710a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:57.122302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803346e-9ba4-444e-b222-47c512a710a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:57.123787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803346e-9ba4-444e-b222-47c512a710a2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:57.125357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 27/55] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:02:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:57.131093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803346e-9ba4-444e-b222-47c512a710a2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:57.188894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 29/56] 10.222.0.30 () {60 vars in 1140 bytes} [Sun Aug 7 12:02:56 2022] POST /identity/v3/projects => generated 298 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:02:57.234534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803346e-9ba4-444e-b222-47c512a710a2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:57.253112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:57.272061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26fe57-3086-48df-800e-c301e83fd8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:57.272688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26fe57-3086-48df-800e-c301e83fd8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:57.272971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26fe57-3086-48df-800e-c301e83fd8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:57.273247 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26fe57-3086-48df-800e-c301e83fd8c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:57.273989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26fe57-3086-48df-800e-c301e83fd8c3 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.domain.id=default, user.id=6e390f818f21433d9d55c5e4c3ba79e5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:02:57.274300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26fe57-3086-48df-800e-c301e83fd8c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:57.275811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26fe57-3086-48df-800e-c301e83fd8c3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:57.285431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 28/57] 10.222.0.30 () {56 vars in 1322 bytes} [Sun Aug 7 12:02:57 2022] GET /identity/v3/role_assignments?user.id=6e390f818f21433d9d55c5e4c3ba79e5&scope.domain.id=default&role.id=f72d3388f13149b483355331380f040b => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:02:59.183358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a0ce7-ee88-4dc8-85ac-9fdbc3112bd2 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:59.183872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a0ce7-ee88-4dc8-85ac-9fdbc3112bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:59.183961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a0ce7-ee88-4dc8-85ac-9fdbc3112bd2 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:59.185238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 30/58] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:02:59.190240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5a7fc-3fb4-4760-be7e-4f60d80be33d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:59.190644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5a7fc-3fb4-4760-be7e-4f60d80be33d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:59.190800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5a7fc-3fb4-4760-be7e-4f60d80be33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:59.209035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-aec5a7fc-3fb4-4760-be7e-4f60d80be33d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:02:59.210222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-aec5a7fc-3fb4-4760-be7e-4f60d80be33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:02:59.277252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 29/59] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:02:59.300469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a0ce7-ee88-4dc8-85ac-9fdbc3112bd2 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:59.399912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a0ce7-ee88-4dc8-85ac-9fdbc3112bd2 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:59.431805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9d67a-a22d-4aef-9c45-3ab618bee5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:59.432415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9d67a-a22d-4aef-9c45-3ab618bee5ed None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:59.432696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9d67a-a22d-4aef-9c45-3ab618bee5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:59.433501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9d67a-a22d-4aef-9c45-3ab618bee5ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:59.434655 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9d67a-a22d-4aef-9c45-3ab618bee5ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:59.436122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9d67a-a22d-4aef-9c45-3ab618bee5ed None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:59.437637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 31/60] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:02:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:02:59.443059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5a7fc-3fb4-4760-be7e-4f60d80be33d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:02:59.540072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5a7fc-3fb4-4760-be7e-4f60d80be33d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:02:59.550985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:02:59.574112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812a2a5-6b73-47ce-a5e5-16774515a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:02:59.574710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812a2a5-6b73-47ce-a5e5-16774515a52c None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:02:59.574989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812a2a5-6b73-47ce-a5e5-16774515a52c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:02:59.575294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812a2a5-6b73-47ce-a5e5-16774515a52c None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:02:59.575927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812a2a5-6b73-47ce-a5e5-16774515a52c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:02:59.577389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812a2a5-6b73-47ce-a5e5-16774515a52c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:02:59.695857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 30/61] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:02:59 2022] POST /identity/v3/users => generated 275 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:02.759593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb4ad-02e0-4190-92cd-1b99b505c9f8 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:02.760636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb4ad-02e0-4190-92cd-1b99b505c9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:02.761801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb4ad-02e0-4190-92cd-1b99b505c9f8 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:02.763384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 32/62] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:02.768603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3b13-d069-4c50-9f7e-0983e8aa4472 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:02.769413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3b13-d069-4c50-9f7e-0983e8aa4472 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:02.769906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3b13-d069-4c50-9f7e-0983e8aa4472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:02.795976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5fab3b13-d069-4c50-9f7e-0983e8aa4472 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:02.797488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5fab3b13-d069-4c50-9f7e-0983e8aa4472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:02.841574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a798b-0520-4244-a887-5871e9890d9e None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:02.841942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a798b-0520-4244-a887-5871e9890d9e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:02.842498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a798b-0520-4244-a887-5871e9890d9e None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:02.844310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 33/63] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:02.864344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b19c65-4b9a-45da-a55a-ac95de48b0a6 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:02.864932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b19c65-4b9a-45da-a55a-ac95de48b0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:02.865405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b19c65-4b9a-45da-a55a-ac95de48b0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:02.877793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 31/64] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:02.904052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab3b13-d069-4c50-9f7e-0983e8aa4472 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:02.909395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-91b19c65-4b9a-45da-a55a-ac95de48b0a6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:02.910855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-91b19c65-4b9a-45da-a55a-ac95de48b0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:02.926241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:03.017881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab3b13-d069-4c50-9f7e-0983e8aa4472 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.027579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:03.079648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 34/65] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:03.088279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a69dba-5fc7-4a1f-9378-2c5483df4277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.089800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69dba-5fc7-4a1f-9378-2c5483df4277 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.090289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69dba-5fc7-4a1f-9378-2c5483df4277 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.090775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69dba-5fc7-4a1f-9378-2c5483df4277 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.103831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b19c65-4b9a-45da-a55a-ac95de48b0a6 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.113943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a69dba-5fc7-4a1f-9378-2c5483df4277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e8a69dba-5fc7-4a1f-9378-2c5483df4277 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:03.120804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:03.121844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:03.122636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 32/66] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:03.130492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a69dba-5fc7-4a1f-9378-2c5483df4277 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.259864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b19c65-4b9a-45da-a55a-ac95de48b0a6 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.276149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a69dba-5fc7-4a1f-9378-2c5483df4277 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.298426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e30c1c-00e8-45e8-aa93-fb176338ba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.299085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e30c1c-00e8-45e8-aa93-fb176338ba70 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.299368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e30c1c-00e8-45e8-aa93-fb176338ba70 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.299644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e30c1c-00e8-45e8-aa93-fb176338ba70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.309996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e30c1c-00e8-45e8-aa93-fb176338ba70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:03.311744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec83804-01fc-4000-8417-4bc37dd44531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.312361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec83804-01fc-4000-8417-4bc37dd44531 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.312645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec83804-01fc-4000-8417-4bc37dd44531 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.312918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec83804-01fc-4000-8417-4bc37dd44531 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.313589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec83804-01fc-4000-8417-4bc37dd44531 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:03.313895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec83804-01fc-4000-8417-4bc37dd44531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-d3e30c1c-00e8-45e8-aa93-fb176338ba70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:03.314708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:03.315734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:03.316518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:03.317202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec83804-01fc-4000-8417-4bc37dd44531 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:03.317275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 35/67] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:03.321868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba486fa-30d2-4315-8911-1467fa343ea0 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.322154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba486fa-30d2-4315-8911-1467fa343ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.324123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba486fa-30d2-4315-8911-1467fa343ea0 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.329506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 36/68] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:03.329506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4ff60-1870-4cc4-a467-0c7078a0fb83 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.329506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4ff60-1870-4cc4-a467-0c7078a0fb83 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.329506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4ff60-1870-4cc4-a467-0c7078a0fb83 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.329506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 37/69] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:03.329918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 33/70] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:03.330002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4ff60-1870-4cc4-a467-0c7078a0fb83 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.333797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded4d9-e163-4c62-83ca-6df7de477157 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.334088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded4d9-e163-4c62-83ca-6df7de477157 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.334367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded4d9-e163-4c62-83ca-6df7de477157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.337526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:03.353459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f1ded4d9-e163-4c62-83ca-6df7de477157 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:03.354668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f1ded4d9-e163-4c62-83ca-6df7de477157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:03.368893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:03.427056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 34/71] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:03.427056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded4d9-e163-4c62-83ca-6df7de477157 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.446960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4ff60-1870-4cc4-a467-0c7078a0fb83 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.487351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854643b-0f7a-4cdb-a0a0-ee8c56669919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.488505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854643b-0f7a-4cdb-a0a0-ee8c56669919 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.488791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854643b-0f7a-4cdb-a0a0-ee8c56669919 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.489342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854643b-0f7a-4cdb-a0a0-ee8c56669919 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.490083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854643b-0f7a-4cdb-a0a0-ee8c56669919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:03.490646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854643b-0f7a-4cdb-a0a0-ee8c56669919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:03.492927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854643b-0f7a-4cdb-a0a0-ee8c56669919 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:03.503624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 38/72] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:03.504849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8164c-69f4-4a10-9b33-59ecbcd3d9fb None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.505145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8164c-69f4-4a10-9b33-59ecbcd3d9fb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.505419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8164c-69f4-4a10-9b33-59ecbcd3d9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.525131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-96e8164c-69f4-4a10-9b33-59ecbcd3d9fb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:03.526411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-96e8164c-69f4-4a10-9b33-59ecbcd3d9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:03.545338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded4d9-e163-4c62-83ca-6df7de477157 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.580075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575639e-c778-4d1a-9003-578430e21d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.580702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575639e-c778-4d1a-9003-578430e21d6c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.580986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575639e-c778-4d1a-9003-578430e21d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.581257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575639e-c778-4d1a-9003-578430e21d6c None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.583863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:03.591904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 39/73] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:03.592739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8164c-69f4-4a10-9b33-59ecbcd3d9fb None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.605753 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575639e-c778-4d1a-9003-578430e21d6c None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:03.607368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575639e-c778-4d1a-9003-578430e21d6c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:03.609065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 35/74] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:03 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:03.609848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575639e-c778-4d1a-9003-578430e21d6c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.619875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:03.695258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8164c-69f4-4a10-9b33-59ecbcd3d9fb None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.711107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575639e-c778-4d1a-9003-578430e21d6c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.728750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac5006-6c2c-487d-94c6-c20f7703ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.729374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac5006-6c2c-487d-94c6-c20f7703ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.729684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac5006-6c2c-487d-94c6-c20f7703ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.729957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac5006-6c2c-487d-94c6-c20f7703ebf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.731096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac5006-6c2c-487d-94c6-c20f7703ebf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:03.732540 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac5006-6c2c-487d-94c6-c20f7703ebf7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:03.734123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 40/75] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:03.735246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772010de-0255-4380-9c58-282a72dee7d9 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.735526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772010de-0255-4380-9c58-282a72dee7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.735800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772010de-0255-4380-9c58-282a72dee7d9 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.737224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 41/76] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:03.741150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772010de-0255-4380-9c58-282a72dee7d9 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.751243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1ff22-84da-4900-a44b-94b21c132e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.751872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ff22-84da-4900-a44b-94b21c132e76 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.752155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ff22-84da-4900-a44b-94b21c132e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.752433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ff22-84da-4900-a44b-94b21c132e76 None admin] PATH_INFO: `/v3/users/6e390f818f21433d9d55c5e4c3ba79e5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.753563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1ff22-84da-4900-a44b-94b21c132e76 None admin] RBAC: Authorizing `identity:get_user(user_id=6e390f818f21433d9d55c5e4c3ba79e5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:03.755009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1ff22-84da-4900-a44b-94b21c132e76 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:03.756596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 36/77] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:03 2022] GET /identity/v3/users/6e390f818f21433d9d55c5e4c3ba79e5 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:03.757420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1ff22-84da-4900-a44b-94b21c132e76 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.803380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:03.832497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:03.842687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772010de-0255-4380-9c58-282a72dee7d9 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.860104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1ff22-84da-4900-a44b-94b21c132e76 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:03.882778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472921a0-ec78-4325-976f-b6cf109bc1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.883414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472921a0-ec78-4325-976f-b6cf109bc1dd None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.883708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472921a0-ec78-4325-976f-b6cf109bc1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.883983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472921a0-ec78-4325-976f-b6cf109bc1dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.895640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f5ee2-3c44-41d4-b0ed-7f9f63f3220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:03.896266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5ee2-3c44-41d4-b0ed-7f9f63f3220a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:03.896790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5ee2-3c44-41d4-b0ed-7f9f63f3220a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:03.896790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5ee2-3c44-41d4-b0ed-7f9f63f3220a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:03.906667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f5ee2-3c44-41d4-b0ed-7f9f63f3220a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-fa1f5ee2-3c44-41d4-b0ed-7f9f63f3220a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:03.911366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:03.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 37/78] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:03.913515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f5ee2-3c44-41d4-b0ed-7f9f63f3220a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:03.956161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472921a0-ec78-4325-976f-b6cf109bc1dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:03.957623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472921a0-ec78-4325-976f-b6cf109bc1dd None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:04.011760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f5ee2-3c44-41d4-b0ed-7f9f63f3220a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:04.016267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:04.045771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d39605-3c7d-4218-b30d-617ac4c80fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:04.046374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d39605-3c7d-4218-b30d-617ac4c80fff None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.046655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d39605-3c7d-4218-b30d-617ac4c80fff None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.046932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d39605-3c7d-4218-b30d-617ac4c80fff None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.047607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d39605-3c7d-4218-b30d-617ac4c80fff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:04.049054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d39605-3c7d-4218-b30d-617ac4c80fff None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:04.055317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 42/79] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:04.056487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72848e92-eba9-4754-b841-1cd51bebc043 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.056784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72848e92-eba9-4754-b841-1cd51bebc043 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.057071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72848e92-eba9-4754-b841-1cd51bebc043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.060108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:04.076581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-72848e92-eba9-4754-b841-1cd51bebc043 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:04.077782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-72848e92-eba9-4754-b841-1cd51bebc043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:04.121710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 38/80] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:03 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:04.122548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d39605-3c7d-4218-b30d-617ac4c80fff None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.149200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 43/81] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:04.150384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72848e92-eba9-4754-b841-1cd51bebc043 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.257139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d39605-3c7d-4218-b30d-617ac4c80fff None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:04.269694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:04.281274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72848e92-eba9-4754-b841-1cd51bebc043 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:04.300947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a385eec-6afa-444f-bfaa-9c2ad6d3c549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:04.301571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a385eec-6afa-444f-bfaa-9c2ad6d3c549 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.301856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a385eec-6afa-444f-bfaa-9c2ad6d3c549 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.302134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a385eec-6afa-444f-bfaa-9c2ad6d3c549 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.322452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:04.323628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.323957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.324251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.324987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:04.325291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:04.326687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:04.335917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 44/82] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:04.336526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.373783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:04.390461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a385eec-6afa-444f-bfaa-9c2ad6d3c549 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:04.391951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a385eec-6afa-444f-bfaa-9c2ad6d3c549 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:04.460398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f290f1d-bbf4-4335-a152-e2b75e0f0166 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:04.504069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7a27-544b-4ef0-b48a-24c1ed0fac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:04.504713 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7a27-544b-4ef0-b48a-24c1ed0fac27 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.504997 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7a27-544b-4ef0-b48a-24c1ed0fac27 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.505274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7a27-544b-4ef0-b48a-24c1ed0fac27 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.506960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab7a27-544b-4ef0-b48a-24c1ed0fac27 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:04.508764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab7a27-544b-4ef0-b48a-24c1ed0fac27 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:04.512087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 45/83] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:04.512932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7a27-544b-4ef0-b48a-24c1ed0fac27 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.522152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 39/84] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:04.523319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a385eec-6afa-444f-bfaa-9c2ad6d3c549 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.603458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:04.638810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:04.650879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a385eec-6afa-444f-bfaa-9c2ad6d3c549 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:04.658135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7a27-544b-4ef0-b48a-24c1ed0fac27 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:04.689423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48fd2e-86cf-4a4c-8d61-9d0477c1d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:04.690436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48fd2e-86cf-4a4c-8d61-9d0477c1d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.690923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48fd2e-86cf-4a4c-8d61-9d0477c1d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.691449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48fd2e-86cf-4a4c-8d61-9d0477c1d2f1 None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.692865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48fd2e-86cf-4a4c-8d61-9d0477c1d2f1 None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:04.694505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48fd2e-86cf-4a4c-8d61-9d0477c1d2f1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:04.696565 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 40/85] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:04.697786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48fd2e-86cf-4a4c-8d61-9d0477c1d2f1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.701166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6c41-353d-4bae-abe9-645ab7223cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:04.702967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6c41-353d-4bae-abe9-645ab7223cda None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.703421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6c41-353d-4bae-abe9-645ab7223cda None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.703833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6c41-353d-4bae-abe9-645ab7223cda None admin] PATH_INFO: `/v3/roles/member` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.717974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba6c41-353d-4bae-abe9-645ab7223cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-3bba6c41-353d-4bae-abe9-645ab7223cda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:04.725566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:04.726609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:04.727632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:04.730934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 46/86] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:04.732565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6c41-353d-4bae-abe9-645ab7223cda None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.828514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48fd2e-86cf-4a4c-8d61-9d0477c1d2f1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:04.833274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:04.852537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6c41-353d-4bae-abe9-645ab7223cda None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:04.878457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:04.879127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.880169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.880456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.881204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] RBAC: Adding query filter params (role.id=347174cd91d94848aec704f00988e871, scope.project.id=5d73bd77f96146698091e497e8e76566, user.id=18a66efba8ca4c218281db23e0303fe5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:04.881521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:04.882985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:04.894233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 41/87] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/role_assignments?user.id=18a66efba8ca4c218281db23e0303fe5&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=347174cd91d94848aec704f00988e871 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:04.896938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.901812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e04b5-285b-4d59-b866-afcea01f1011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:04.902743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e04b5-285b-4d59-b866-afcea01f1011 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:04.903091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e04b5-285b-4d59-b866-afcea01f1011 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:04.903473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e04b5-285b-4d59-b866-afcea01f1011 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:04.905792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e04b5-285b-4d59-b866-afcea01f1011 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:04.907615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e04b5-285b-4d59-b866-afcea01f1011 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:04.910389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 47/88] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:04.915141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e04b5-285b-4d59-b866-afcea01f1011 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:04.940777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:05.066766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e04b5-285b-4d59-b866-afcea01f1011 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:05.069267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de361ef2-2da6-44a0-ad4b-7a1b4e7238de None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:05.112958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:05.113704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:05.113986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:05.114256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:05.114928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:05.115259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:05.116890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:05.127164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 48/89] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:05.128000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:05.141933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62467c20-ebc4-4a55-a7ad-65b4c23e3320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:05.142850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62467c20-ebc4-4a55-a7ad-65b4c23e3320 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:05.143426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62467c20-ebc4-4a55-a7ad-65b4c23e3320 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:05.143927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62467c20-ebc4-4a55-a7ad-65b4c23e3320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:05.146028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:05.250075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42117f69-d265-4271-a76d-20ce6faf80ae None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:05.252660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62467c20-ebc4-4a55-a7ad-65b4c23e3320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:05.254386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62467c20-ebc4-4a55-a7ad-65b4c23e3320 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:05.288294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:05.288906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:05.289185 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:05.289456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:05.290569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=5d73bd77f96146698091e497e8e76566, user.id=6e390f818f21433d9d55c5e4c3ba79e5) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:05.290882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:05.292509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:05.300251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 49/90] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:05 2022] GET /identity/v3/role_assignments?user.id=6e390f818f21433d9d55c5e4c3ba79e5&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=f72d3388f13149b483355331380f040b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:05.300251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:05.312332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:05.472634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 42/91] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:05.479047 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62467c20-ebc4-4a55-a7ad-65b4c23e3320 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:05.496229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:05.547115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ed787-2bf7-4882-b6c6-6ae3c2df0824 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:05.616457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586513bf-ed00-417a-b881-e2e0b2cfcae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:05.617283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586513bf-ed00-417a-b881-e2e0b2cfcae9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:05.617673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586513bf-ed00-417a-b881-e2e0b2cfcae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:05.618058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586513bf-ed00-417a-b881-e2e0b2cfcae9 None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:05.624311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586513bf-ed00-417a-b881-e2e0b2cfcae9 None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:05.626037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586513bf-ed00-417a-b881-e2e0b2cfcae9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:05.628177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 50/92] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:05 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:05.633051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586513bf-ed00-417a-b881-e2e0b2cfcae9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:05.675131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62467c20-ebc4-4a55-a7ad-65b4c23e3320 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:05.711597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:05.733484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e1ee6-0ddc-4819-839b-6d447188b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:05.734363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1ee6-0ddc-4819-839b-6d447188b635 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:05.735795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1ee6-0ddc-4819-839b-6d447188b635 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:05.736377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1ee6-0ddc-4819-839b-6d447188b635 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:05.738496 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e1ee6-0ddc-4819-839b-6d447188b635 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:05.740291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e1ee6-0ddc-4819-839b-6d447188b635 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:05.742870 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 43/93] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:05 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:05.749614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e1ee6-0ddc-4819-839b-6d447188b635 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:05.822936 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586513bf-ed00-417a-b881-e2e0b2cfcae9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:05.882534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17285-56d0-4254-bdb8-24aaf4afd799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:05.883317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17285-56d0-4254-bdb8-24aaf4afd799 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:05.883507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17285-56d0-4254-bdb8-24aaf4afd799 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:05.883789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17285-56d0-4254-bdb8-24aaf4afd799 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:05.910692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:05.989521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e1ee6-0ddc-4819-839b-6d447188b635 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:06.015095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e17285-56d0-4254-bdb8-24aaf4afd799 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:06.016597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e17285-56d0-4254-bdb8-24aaf4afd799 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:06.049353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:06.049967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:06.050255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:06.050533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:06.051302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] RBAC: Adding query filter params (role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=5d73bd77f96146698091e497e8e76566, user.id=18a66efba8ca4c218281db23e0303fe5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:06.051619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:06.053061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:06.059634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 44/94] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:05 2022] GET /identity/v3/role_assignments?user.id=18a66efba8ca4c218281db23e0303fe5&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:06.168175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 51/95] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:03:06.176133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:06.274109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:06.329204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8851344-60e8-458a-9728-62dfbc026b8c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:06.365925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaee558-cfae-4309-8c9e-bc23f316558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:06.366567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaee558-cfae-4309-8c9e-bc23f316558c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:06.366860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaee558-cfae-4309-8c9e-bc23f316558c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:06.367170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaee558-cfae-4309-8c9e-bc23f316558c None admin] PATH_INFO: `/v3/projects/1343820eab844e92a91697087f8c9830` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:06.368331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaee558-cfae-4309-8c9e-bc23f316558c None admin] RBAC: Authorizing `identity:get_project(project_id=1343820eab844e92a91697087f8c9830)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:06.369782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaee558-cfae-4309-8c9e-bc23f316558c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:06.372032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 45/96] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:06 2022] GET /identity/v3/projects/1343820eab844e92a91697087f8c9830 => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:06.377943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17285-56d0-4254-bdb8-24aaf4afd799 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:06.436892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:06.516474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17285-56d0-4254-bdb8-24aaf4afd799 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:06.588391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b7894-c705-4fe6-8697-ccaca2d74e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:06.589413 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b7894-c705-4fe6-8697-ccaca2d74e46 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:06.589782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b7894-c705-4fe6-8697-ccaca2d74e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:06.590305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b7894-c705-4fe6-8697-ccaca2d74e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:06.591083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b7894-c705-4fe6-8697-ccaca2d74e46 None admin] RBAC: Adding query filter params (role.id=347174cd91d94848aec704f00988e871, scope.project.id=1343820eab844e92a91697087f8c9830, user.id=18a66efba8ca4c218281db23e0303fe5) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:06.591406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b7894-c705-4fe6-8697-ccaca2d74e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:06.592879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b7894-c705-4fe6-8697-ccaca2d74e46 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:06.598701 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 52/97] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:06 2022] GET /identity/v3/role_assignments?user.id=18a66efba8ca4c218281db23e0303fe5&scope.project.id=1343820eab844e92a91697087f8c9830&role.id=347174cd91d94848aec704f00988e871 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:06.831732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23df03b-b0ea-4428-a42b-318c95ef681e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:06.832022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23df03b-b0ea-4428-a42b-318c95ef681e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:06.832308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23df03b-b0ea-4428-a42b-318c95ef681e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:06.833719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 46/98] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:06.839664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573ddfa-a12f-4ff9-94bb-572005eae5c7 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:06.839985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573ddfa-a12f-4ff9-94bb-572005eae5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:06.840481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573ddfa-a12f-4ff9-94bb-572005eae5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:06.860608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2573ddfa-a12f-4ff9-94bb-572005eae5c7 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:06.861838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2573ddfa-a12f-4ff9-94bb-572005eae5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:06.938805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 53/99] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:06.972090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23df03b-b0ea-4428-a42b-318c95ef681e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:07.054664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515655c3-4de8-4610-b711-2ee7b5fb4702 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:07.054977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515655c3-4de8-4610-b711-2ee7b5fb4702 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:07.055257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515655c3-4de8-4610-b711-2ee7b5fb4702 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:07.056604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 54/100] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:07.062106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57250224-0922-489f-8a3a-abdd9891d549 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:07.062453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57250224-0922-489f-8a3a-abdd9891d549 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:07.062741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57250224-0922-489f-8a3a-abdd9891d549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:07.091853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-57250224-0922-489f-8a3a-abdd9891d549 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:07.092986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-57250224-0922-489f-8a3a-abdd9891d549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:07.150838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23df03b-b0ea-4428-a42b-318c95ef681e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:07.152008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:07.192680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:07.207049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16598a14-6cac-4122-841b-3ac9778ffccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:07.207916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16598a14-6cac-4122-841b-3ac9778ffccd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:07.208348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16598a14-6cac-4122-841b-3ac9778ffccd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:07.208802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16598a14-6cac-4122-841b-3ac9778ffccd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:07.228307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16598a14-6cac-4122-841b-3ac9778ffccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-16598a14-6cac-4122-841b-3ac9778ffccd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:07.235519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:07.236589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:07.237828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:07.240206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 55/101] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:07.241621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 47/102] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:07.247823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57250224-0922-489f-8a3a-abdd9891d549 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:07.274771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16598a14-6cac-4122-841b-3ac9778ffccd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:07.378178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57250224-0922-489f-8a3a-abdd9891d549 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:07.415566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:07.416483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:07.416947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:07.417489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:07.418665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:07.419202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:07.420877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:07.429907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 56/103] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:07.436036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:07.480138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:07.485160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16598a14-6cac-4122-841b-3ac9778ffccd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:07.563738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06c6f-cf05-4ba7-bfbd-8a233c800187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:07.564725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06c6f-cf05-4ba7-bfbd-8a233c800187 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:07.565190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06c6f-cf05-4ba7-bfbd-8a233c800187 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:07.565642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06c6f-cf05-4ba7-bfbd-8a233c800187 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:07.567269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe06c6f-cf05-4ba7-bfbd-8a233c800187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:07.570113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe06c6f-cf05-4ba7-bfbd-8a233c800187 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:07.575979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 48/104] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:07.578752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06c6f-cf05-4ba7-bfbd-8a233c800187 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:07.584906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c6ce7-ca69-477e-b257-83888d8d16ec None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:07.652422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37c4a3-44e9-4d84-b494-ded8abd68d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:07.653530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37c4a3-44e9-4d84-b494-ded8abd68d85 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:07.654903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37c4a3-44e9-4d84-b494-ded8abd68d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:07.655978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37c4a3-44e9-4d84-b494-ded8abd68d85 None admin] PATH_INFO: `/v3/users/01a2cd5391ee4c44923895da0d75c715` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:07.684205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37c4a3-44e9-4d84-b494-ded8abd68d85 None admin] RBAC: Authorizing `identity:get_user(user_id=01a2cd5391ee4c44923895da0d75c715)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:07.687369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:07.688883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37c4a3-44e9-4d84-b494-ded8abd68d85 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:07.691920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 57/105] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:07 2022] GET /identity/v3/users/01a2cd5391ee4c44923895da0d75c715 => generated 289 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:07.697249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37c4a3-44e9-4d84-b494-ded8abd68d85 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:07.805559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37c4a3-44e9-4d84-b494-ded8abd68d85 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:07.809216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06c6f-cf05-4ba7-bfbd-8a233c800187 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:07.834304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:07.843987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022226a-9382-4690-8f73-57a64776f771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:07.845213 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022226a-9382-4690-8f73-57a64776f771 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:07.845728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022226a-9382-4690-8f73-57a64776f771 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:07.846188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022226a-9382-4690-8f73-57a64776f771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:07.925060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022226a-9382-4690-8f73-57a64776f771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:07.926915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022226a-9382-4690-8f73-57a64776f771 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:07.931387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20248af3-79eb-4dbf-aab8-26687dc29faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:07.932009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20248af3-79eb-4dbf-aab8-26687dc29faa None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:07.932295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20248af3-79eb-4dbf-aab8-26687dc29faa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:07.932572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20248af3-79eb-4dbf-aab8-26687dc29faa None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:07.933309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20248af3-79eb-4dbf-aab8-26687dc29faa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:07.934742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20248af3-79eb-4dbf-aab8-26687dc29faa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:08.071388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:08.077761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 58/106] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:03:08.079255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37beb-b6eb-4750-9044-c0feb97a41d0 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.079829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37beb-b6eb-4750-9044-c0feb97a41d0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.080302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37beb-b6eb-4750-9044-c0feb97a41d0 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.081762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 59/107] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:08.084000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37beb-b6eb-4750-9044-c0feb97a41d0 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:08.085171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 49/108] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:03:07 2022] POST /identity/v3/projects => generated 302 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:08.092597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ae1c-2ecb-48c7-940a-b4b9470bfa7a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.092893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ae1c-2ecb-48c7-940a-b4b9470bfa7a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.093153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ae1c-2ecb-48c7-940a-b4b9470bfa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.139368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7a75ae1c-2ecb-48c7-940a-b4b9470bfa7a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:08.140865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7a75ae1c-2ecb-48c7-940a-b4b9470bfa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:08.208458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37beb-b6eb-4750-9044-c0feb97a41d0 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:08.249217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b211a0-2364-4cea-988c-a9a03138bb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:08.250272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b211a0-2364-4cea-988c-a9a03138bb4b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.250760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b211a0-2364-4cea-988c-a9a03138bb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.251243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b211a0-2364-4cea-988c-a9a03138bb4b None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.252742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b211a0-2364-4cea-988c-a9a03138bb4b None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:08.254436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b211a0-2364-4cea-988c-a9a03138bb4b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:08.256902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 60/109] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:08 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:08.264205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b211a0-2364-4cea-988c-a9a03138bb4b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:08.306783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 50/110] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:08.310341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c69b9-9a2e-4750-9cca-9793f16198c9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.310341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c69b9-9a2e-4750-9cca-9793f16198c9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.310341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c69b9-9a2e-4750-9cca-9793f16198c9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.310341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 51/111] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:08.315070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1718-8dab-4496-a9a9-5405e4833f80 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.315356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1718-8dab-4496-a9a9-5405e4833f80 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.315634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1718-8dab-4496-a9a9-5405e4833f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.334611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e9cf1718-8dab-4496-a9a9-5405e4833f80 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:08.335837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e9cf1718-8dab-4496-a9a9-5405e4833f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:08.360585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:08.375529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:08.408079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b211a0-2364-4cea-988c-a9a03138bb4b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:08.435546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 52/112] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:08.436851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf1718-8dab-4496-a9a9-5405e4833f80 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:08.458406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:08.459089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.459331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.459722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.460506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] RBAC: Adding query filter params (role.id=a5998ecc28d44e0c9deb0653af1c8eaf, scope.project.id=5d73bd77f96146698091e497e8e76566, user.id=01a2cd5391ee4c44923895da0d75c715) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:08.460820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:08.462262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:08.475492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 61/113] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:08 2022] GET /identity/v3/role_assignments?user.id=01a2cd5391ee4c44923895da0d75c715&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=a5998ecc28d44e0c9deb0653af1c8eaf => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:08.499715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:08.642338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf1718-8dab-4496-a9a9-5405e4833f80 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:08.692145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c22a8-b821-4dcd-81d2-dd34032440a8 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:08.715783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:08.728653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab29b2a-8bcd-4204-98bb-314481b640c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:08.729401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab29b2a-8bcd-4204-98bb-314481b640c5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.730293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab29b2a-8bcd-4204-98bb-314481b640c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.730641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab29b2a-8bcd-4204-98bb-314481b640c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.733512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a47a31-f371-4427-9a3a-aed7ff3ac664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:08.734406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a47a31-f371-4427-9a3a-aed7ff3ac664 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.734870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a47a31-f371-4427-9a3a-aed7ff3ac664 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.735467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a47a31-f371-4427-9a3a-aed7ff3ac664 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.749013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a47a31-f371-4427-9a3a-aed7ff3ac664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-88a47a31-f371-4427-9a3a-aed7ff3ac664 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:08.754131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:08.755056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:08.755854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:08.756486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab29b2a-8bcd-4204-98bb-314481b640c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:08.758023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 62/114] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9ab29b2a-8bcd-4204-98bb-314481b640c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:08.762037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:08.763117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:08.764193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:08.765765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 53/115] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:08.766281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a47a31-f371-4427-9a3a-aed7ff3ac664 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:08.784509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab29b2a-8bcd-4204-98bb-314481b640c5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:08.845864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:08.889327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a47a31-f371-4427-9a3a-aed7ff3ac664 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:08.930827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:08.931778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:08.932259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:08.933254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:08.934273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:08.934730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:08.936412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:08.946221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 63/116] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:08.951652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:09.026018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab29b2a-8bcd-4204-98bb-314481b640c5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:09.043453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:09.100761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:09.101840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.102271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.102692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.103605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:09.104102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:09.106347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:09.130671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 54/117] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:09.140087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:09.197171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54272fce-6ae6-46c0-b4dc-f1cc7450c1df None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:09.272651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:09.276860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b3874-5889-4318-950d-5f26a0003686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:09.277638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b3874-5889-4318-950d-5f26a0003686 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.277921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b3874-5889-4318-950d-5f26a0003686 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.278200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b3874-5889-4318-950d-5f26a0003686 None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.279393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b3874-5889-4318-950d-5f26a0003686 None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:09.280863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b3874-5889-4318-950d-5f26a0003686 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:09.282530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 64/118] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:08 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:09.292560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b3874-5889-4318-950d-5f26a0003686 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:09.296187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acfc91-bae9-4c68-87fe-f2cd84479525 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:09.367494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac949093-f304-458f-aaaa-ffeb1078ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:09.368159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac949093-f304-458f-aaaa-ffeb1078ac89 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.368506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac949093-f304-458f-aaaa-ffeb1078ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.368792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac949093-f304-458f-aaaa-ffeb1078ac89 None admin] PATH_INFO: `/v3/users/6e390f818f21433d9d55c5e4c3ba79e5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.374032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac949093-f304-458f-aaaa-ffeb1078ac89 None admin] RBAC: Authorizing `identity:get_user(user_id=6e390f818f21433d9d55c5e4c3ba79e5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:09.374032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac949093-f304-458f-aaaa-ffeb1078ac89 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:09.374032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 55/119] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:09 2022] GET /identity/v3/users/6e390f818f21433d9d55c5e4c3ba79e5 => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:09.375812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316549-ab39-44ef-a102-41a289a6bf1f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.376325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316549-ab39-44ef-a102-41a289a6bf1f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.377325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316549-ab39-44ef-a102-41a289a6bf1f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.383579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 56/120] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:09.384597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb316549-ab39-44ef-a102-41a289a6bf1f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:09.470692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b3874-5889-4318-950d-5f26a0003686 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:09.485111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:09.518051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b538912-0215-4329-999b-887f644bc40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:09.518696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b538912-0215-4329-999b-887f644bc40c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.518982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b538912-0215-4329-999b-887f644bc40c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.519292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b538912-0215-4329-999b-887f644bc40c None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.521026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b538912-0215-4329-999b-887f644bc40c None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:09.522756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b538912-0215-4329-999b-887f644bc40c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:09.525453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 65/121] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:09 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:09.527828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17141d2c-be45-4015-a19b-05b5a18b8a15 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.528176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17141d2c-be45-4015-a19b-05b5a18b8a15 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.528466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17141d2c-be45-4015-a19b-05b5a18b8a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.538425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb316549-ab39-44ef-a102-41a289a6bf1f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:09.550043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-17141d2c-be45-4015-a19b-05b5a18b8a15 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:09.551603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-17141d2c-be45-4015-a19b-05b5a18b8a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:09.576081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942782c5-f173-4a1c-8440-485e55997c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:09.576718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942782c5-f173-4a1c-8440-485e55997c89 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.577003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942782c5-f173-4a1c-8440-485e55997c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.577283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942782c5-f173-4a1c-8440-485e55997c89 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.578413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942782c5-f173-4a1c-8440-485e55997c89 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:09.580003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942782c5-f173-4a1c-8440-485e55997c89 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:09.581822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 57/122] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:09 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:09.582965 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2ff0-a8a5-4a4a-8ca6-4cfd0a3e88ec None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.583293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2ff0-a8a5-4a4a-8ca6-4cfd0a3e88ec None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.583576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2ff0-a8a5-4a4a-8ca6-4cfd0a3e88ec None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.589473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 58/123] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:09.589473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d2ff0-a8a5-4a4a-8ca6-4cfd0a3e88ec None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:09.614918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:09.641861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 66/124] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:09.643209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17141d2c-be45-4015-a19b-05b5a18b8a15 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:09.702437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d2ff0-a8a5-4a4a-8ca6-4cfd0a3e88ec None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:09.738428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ade2e4-17c5-403c-83c3-1a85c400e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:09.739129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade2e4-17c5-403c-83c3-1a85c400e489 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.739413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade2e4-17c5-403c-83c3-1a85c400e489 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.739692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade2e4-17c5-403c-83c3-1a85c400e489 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566/users/18a66efba8ca4c218281db23e0303fe5/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.741596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ade2e4-17c5-403c-83c3-1a85c400e489 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d73bd77f96146698091e497e8e76566, user_id=18a66efba8ca4c218281db23e0303fe5, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:09.743050 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ade2e4-17c5-403c-83c3-1a85c400e489 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:09.762106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17141d2c-be45-4015-a19b-05b5a18b8a15 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:09.772749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:09.798457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9dfae-4270-4838-8233-47ae8c19ad56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:09.799072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9dfae-4270-4838-8233-47ae8c19ad56 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.799358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9dfae-4270-4838-8233-47ae8c19ad56 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.799636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9dfae-4270-4838-8233-47ae8c19ad56 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566/users/6e390f818f21433d9d55c5e4c3ba79e5/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.801518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9dfae-4270-4838-8233-47ae8c19ad56 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d73bd77f96146698091e497e8e76566, user_id=6e390f818f21433d9d55c5e4c3ba79e5, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:09.803041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9dfae-4270-4838-8233-47ae8c19ad56 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:09.818326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 59/125] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:09 2022] PUT /identity/v3/projects/5d73bd77f96146698091e497e8e76566/users/18a66efba8ca4c218281db23e0303fe5/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:09.818326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e003fad-685b-4a2d-b23d-b44f903b7d3e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:09.818326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e003fad-685b-4a2d-b23d-b44f903b7d3e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:09.818618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e003fad-685b-4a2d-b23d-b44f903b7d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:09.840858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6e003fad-685b-4a2d-b23d-b44f903b7d3e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:09.842170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6e003fad-685b-4a2d-b23d-b44f903b7d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:09.867685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 67/126] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:09 2022] PUT /identity/v3/projects/5d73bd77f96146698091e497e8e76566/users/6e390f818f21433d9d55c5e4c3ba79e5/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:09.868658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9dfae-4270-4838-8233-47ae8c19ad56 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:09.954566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 60/127] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:10.036191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e003fad-685b-4a2d-b23d-b44f903b7d3e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:10.051329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:10.054074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9dfae-4270-4838-8233-47ae8c19ad56 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:10.102651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4bca5-57c1-49ea-978c-e7526dd59c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:10.103335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4bca5-57c1-49ea-978c-e7526dd59c85 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:10.103620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4bca5-57c1-49ea-978c-e7526dd59c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:10.103899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4bca5-57c1-49ea-978c-e7526dd59c85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:10.121505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4bca5-57c1-49ea-978c-e7526dd59c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-21d4bca5-57c1-49ea-978c-e7526dd59c85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:10.139346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:10.140310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:10.141101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 68/128] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:10.146540 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4bca5-57c1-49ea-978c-e7526dd59c85 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:10.191457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:10.267040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e003fad-685b-4a2d-b23d-b44f903b7d3e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:10.288382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4bca5-57c1-49ea-978c-e7526dd59c85 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:10.329199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:10.329728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:10.330013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:10.330293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:10.331151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:10.331469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:10.333073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:10.340669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074d75f-bb72-466c-bceb-72d9b2ddc62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:10.341441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074d75f-bb72-466c-bceb-72d9b2ddc62f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:10.341822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074d75f-bb72-466c-bceb-72d9b2ddc62f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:10.342195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074d75f-bb72-466c-bceb-72d9b2ddc62f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:10.343451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 69/129] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:10.352184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:10.371732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074d75f-bb72-466c-bceb-72d9b2ddc62f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2074d75f-bb72-466c-bceb-72d9b2ddc62f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:10.376677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:10.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:10.378714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:10.381347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 61/130] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:10.386459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074d75f-bb72-466c-bceb-72d9b2ddc62f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:10.514228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:10.537015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d07121-8625-427b-955b-f8be7b8b8798 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:10.560958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:10.608663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89701-1845-45b1-9662-1d2d347d0888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:10.609244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89701-1845-45b1-9662-1d2d347d0888 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:10.609513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89701-1845-45b1-9662-1d2d347d0888 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:10.609796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89701-1845-45b1-9662-1d2d347d0888 None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:10.612017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec89701-1845-45b1-9662-1d2d347d0888 None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:10.613517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec89701-1845-45b1-9662-1d2d347d0888 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:10.615214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 70/131] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:10 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:10.620907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074d75f-bb72-466c-bceb-72d9b2ddc62f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:10.621423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89701-1845-45b1-9662-1d2d347d0888 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:10.696443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:10.697058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:10.697338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:10.697607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:10.698888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:10.699361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:10.700759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:10.718084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 62/132] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:10.724746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:10.783099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89701-1845-45b1-9662-1d2d347d0888 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:10.818311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c373e-0ced-4d3e-a21c-0571ad2c7004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:10.818948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c373e-0ced-4d3e-a21c-0571ad2c7004 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:10.819262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c373e-0ced-4d3e-a21c-0571ad2c7004 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:10.819541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c373e-0ced-4d3e-a21c-0571ad2c7004 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:10.820769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c373e-0ced-4d3e-a21c-0571ad2c7004 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:10.822236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c373e-0ced-4d3e-a21c-0571ad2c7004 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:10.824886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 71/133] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:10 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:10.830263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c373e-0ced-4d3e-a21c-0571ad2c7004 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:10.860510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:10.935474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c373e-0ced-4d3e-a21c-0571ad2c7004 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:10.972011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dd62e-cd23-414c-a24b-b0c38e35e9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:10.972676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dd62e-cd23-414c-a24b-b0c38e35e9cb None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:10.972962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dd62e-cd23-414c-a24b-b0c38e35e9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:10.973241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dd62e-cd23-414c-a24b-b0c38e35e9cb None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566/users/18a66efba8ca4c218281db23e0303fe5/roles/9475450bfdd04912abb88ccf9973f9d8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:10.975464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4dd62e-cd23-414c-a24b-b0c38e35e9cb None admin] RBAC: Authorizing `identity:create_grant(project_id=5d73bd77f96146698091e497e8e76566, user_id=18a66efba8ca4c218281db23e0303fe5, role_id=9475450bfdd04912abb88ccf9973f9d8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:10.977034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4dd62e-cd23-414c-a24b-b0c38e35e9cb None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:10.991165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e34f-708e-492f-aa6d-615b9e7d8d75 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:11.027060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:11.038231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8481fe8-b1ca-47b2-98b3-54bd67adef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:11.039337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481fe8-b1ca-47b2-98b3-54bd67adef20 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.039819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481fe8-b1ca-47b2-98b3-54bd67adef20 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.040317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481fe8-b1ca-47b2-98b3-54bd67adef20 None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.041839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8481fe8-b1ca-47b2-98b3-54bd67adef20 None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:11.044200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8481fe8-b1ca-47b2-98b3-54bd67adef20 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:11.046012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 63/134] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:10 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:11.053515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8481fe8-b1ca-47b2-98b3-54bd67adef20 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:11.062434 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 72/135] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:10 2022] PUT /identity/v3/projects/5d73bd77f96146698091e497e8e76566/users/18a66efba8ca4c218281db23e0303fe5/roles/9475450bfdd04912abb88ccf9973f9d8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:03:11.109515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4840-3031-46dc-ab69-b143674ec8cd None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.109906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4840-3031-46dc-ab69-b143674ec8cd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.110094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4840-3031-46dc-ab69-b143674ec8cd None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.125319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 73/136] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:11 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:11.131059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35811d8-fd28-425c-a8a3-448c6390e959 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.131345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35811d8-fd28-425c-a8a3-448c6390e959 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.131622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35811d8-fd28-425c-a8a3-448c6390e959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.176834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8481fe8-b1ca-47b2-98b3-54bd67adef20 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:11.180739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c35811d8-fd28-425c-a8a3-448c6390e959 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:11.183409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c35811d8-fd28-425c-a8a3-448c6390e959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:11.222534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3105d-0972-4604-a59a-fb0fd99a33d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:11.223519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3105d-0972-4604-a59a-fb0fd99a33d7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.223982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3105d-0972-4604-a59a-fb0fd99a33d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.224462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3105d-0972-4604-a59a-fb0fd99a33d7 None admin] PATH_INFO: `/v3/projects/1343820eab844e92a91697087f8c9830` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.227078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3105d-0972-4604-a59a-fb0fd99a33d7 None admin] RBAC: Authorizing `identity:get_project(project_id=1343820eab844e92a91697087f8c9830)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:11.228774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3105d-0972-4604-a59a-fb0fd99a33d7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:11.231566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 64/137] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:11 2022] GET /identity/v3/projects/1343820eab844e92a91697087f8c9830 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:11.237799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3105d-0972-4604-a59a-fb0fd99a33d7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:11.338205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:11.377895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3105d-0972-4604-a59a-fb0fd99a33d7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:11.396631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 74/138] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:11.425079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8cfcb-d265-4c99-896d-433ac6de371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:11.426087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8cfcb-d265-4c99-896d-433ac6de371b None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.426550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8cfcb-d265-4c99-896d-433ac6de371b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.427049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8cfcb-d265-4c99-896d-433ac6de371b None admin] PATH_INFO: `/v3/projects/1343820eab844e92a91697087f8c9830/users/18a66efba8ca4c218281db23e0303fe5/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.431121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35811d8-fd28-425c-a8a3-448c6390e959 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:11.431379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8cfcb-d265-4c99-896d-433ac6de371b None admin] RBAC: Authorizing `identity:create_grant(project_id=1343820eab844e92a91697087f8c9830, user_id=18a66efba8ca4c218281db23e0303fe5, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:11.433764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8cfcb-d265-4c99-896d-433ac6de371b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:11.458607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:11.513272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 65/139] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:11 2022] PUT /identity/v3/projects/1343820eab844e92a91697087f8c9830/users/18a66efba8ca4c218281db23e0303fe5/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:03:11.574567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35811d8-fd28-425c-a8a3-448c6390e959 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:11.619904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0feff-7a5f-426d-9bc3-b91ab4e68408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:11.621141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0feff-7a5f-426d-9bc3-b91ab4e68408 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.621420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0feff-7a5f-426d-9bc3-b91ab4e68408 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.621693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0feff-7a5f-426d-9bc3-b91ab4e68408 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.627028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0feff-7a5f-426d-9bc3-b91ab4e68408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:11.628784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0feff-7a5f-426d-9bc3-b91ab4e68408 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:11.630924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 75/140] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:11.637242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8cfcb-d265-4c99-896d-433ac6de371b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:11.683030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9433cb-b81d-4726-a57e-aceccf70c135 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.683030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9433cb-b81d-4726-a57e-aceccf70c135 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.683030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9433cb-b81d-4726-a57e-aceccf70c135 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.683344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 76/141] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:11.689477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b91908-779b-4cbe-a615-16f24df702f1 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.689770 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b91908-779b-4cbe-a615-16f24df702f1 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.690036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b91908-779b-4cbe-a615-16f24df702f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.728034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a0b91908-779b-4cbe-a615-16f24df702f1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:11.729697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a0b91908-779b-4cbe-a615-16f24df702f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:11.770190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8cfcb-d265-4c99-896d-433ac6de371b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:11.808425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:11.813962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27871e59-9e78-41b2-9197-5c6b51454f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:11.814844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27871e59-9e78-41b2-9197-5c6b51454f83 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:11.815357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27871e59-9e78-41b2-9197-5c6b51454f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:11.815909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27871e59-9e78-41b2-9197-5c6b51454f83 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:11.816877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27871e59-9e78-41b2-9197-5c6b51454f83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:11.818524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27871e59-9e78-41b2-9197-5c6b51454f83 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:11.873725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 77/142] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:11.900154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b91908-779b-4cbe-a615-16f24df702f1 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:11.920665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 66/143] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:11 2022] POST /identity/v3/users => generated 283 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:11.927680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:12.032495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b91908-779b-4cbe-a615-16f24df702f1 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:12.080267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652ed4-b771-419d-b900-ed81e34a2283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:12.081066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652ed4-b771-419d-b900-ed81e34a2283 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.081350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652ed4-b771-419d-b900-ed81e34a2283 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.081621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652ed4-b771-419d-b900-ed81e34a2283 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.102407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b652ed4-b771-419d-b900-ed81e34a2283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-4b652ed4-b771-419d-b900-ed81e34a2283 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:12.107408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:12.108324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:12.109604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 78/144] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:12.115282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27871e59-9e78-41b2-9197-5c6b51454f83 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:12.251196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27871e59-9e78-41b2-9197-5c6b51454f83 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:12.293754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:12.294390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.294672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.294945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.295657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:12.295960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:12.297331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:12.309394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 67/145] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:12.315114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652ed4-b771-419d-b900-ed81e34a2283 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:12.406387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:12.441909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652ed4-b771-419d-b900-ed81e34a2283 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:12.482587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56b2c0-2d94-4fca-ade2-1abe8bfc7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:12.483254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56b2c0-2d94-4fca-ade2-1abe8bfc7a79 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.483580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56b2c0-2d94-4fca-ade2-1abe8bfc7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.483857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56b2c0-2d94-4fca-ade2-1abe8bfc7a79 None admin] PATH_INFO: `/v3/users/01a2cd5391ee4c44923895da0d75c715` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.487948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56b2c0-2d94-4fca-ade2-1abe8bfc7a79 None admin] RBAC: Authorizing `identity:get_user(user_id=01a2cd5391ee4c44923895da0d75c715)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:12.489501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56b2c0-2d94-4fca-ade2-1abe8bfc7a79 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:12.491526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 79/146] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:12 2022] GET /identity/v3/users/01a2cd5391ee4c44923895da0d75c715 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:12.497063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:12.578712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:12.644806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95095ff3-0de8-4a56-991c-fd71eb26983b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:12.688603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469806a-85fe-470a-b5db-13bc9271fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:12.689256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469806a-85fe-470a-b5db-13bc9271fd5e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.689541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469806a-85fe-470a-b5db-13bc9271fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.689817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469806a-85fe-470a-b5db-13bc9271fd5e None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.690976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469806a-85fe-470a-b5db-13bc9271fd5e None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:12.692446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469806a-85fe-470a-b5db-13bc9271fd5e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:12.694084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 68/147] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:12 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:12.700489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56b2c0-2d94-4fca-ade2-1abe8bfc7a79 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:12.847808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56b2c0-2d94-4fca-ade2-1abe8bfc7a79 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:12.889769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817aeb35-ff18-4deb-a0ee-be44c7512278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:12.890437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817aeb35-ff18-4deb-a0ee-be44c7512278 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.890718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817aeb35-ff18-4deb-a0ee-be44c7512278 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.890993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817aeb35-ff18-4deb-a0ee-be44c7512278 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566/users/01a2cd5391ee4c44923895da0d75c715/roles/a5998ecc28d44e0c9deb0653af1c8eaf` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.893068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817aeb35-ff18-4deb-a0ee-be44c7512278 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d73bd77f96146698091e497e8e76566, user_id=01a2cd5391ee4c44923895da0d75c715, role_id=a5998ecc28d44e0c9deb0653af1c8eaf)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:12.894534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817aeb35-ff18-4deb-a0ee-be44c7512278 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:12.920356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33b5f1-df78-4994-9cad-2ba261fad5e3 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.920762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33b5f1-df78-4994-9cad-2ba261fad5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.920989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33b5f1-df78-4994-9cad-2ba261fad5e3 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.922342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 69/148] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:12.924322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec0f8-83f0-4c5e-8cb5-77c9f1214002 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.924605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec0f8-83f0-4c5e-8cb5-77c9f1214002 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.924878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec0f8-83f0-4c5e-8cb5-77c9f1214002 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.926073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 70/149] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:12.932351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406ada2-0cfd-47d6-abcc-7dc434b52989 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.933162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406ada2-0cfd-47d6-abcc-7dc434b52989 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.933443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406ada2-0cfd-47d6-abcc-7dc434b52989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.959297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a406ada2-0cfd-47d6-abcc-7dc434b52989 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:12.960571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a406ada2-0cfd-47d6-abcc-7dc434b52989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:12.973354 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 80/150] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:12 2022] PUT /identity/v3/projects/5d73bd77f96146698091e497e8e76566/users/01a2cd5391ee4c44923895da0d75c715/roles/a5998ecc28d44e0c9deb0653af1c8eaf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:12.974945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395289e8-a7f1-4ed6-8f24-6994bbc0d87f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:12.975254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395289e8-a7f1-4ed6-8f24-6994bbc0d87f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:12.975524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395289e8-a7f1-4ed6-8f24-6994bbc0d87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:12.984045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:13.000931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-395289e8-a7f1-4ed6-8f24-6994bbc0d87f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:13.002325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-395289e8-a7f1-4ed6-8f24-6994bbc0d87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:13.091995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 71/151] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:13.125705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 81/152] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:13.143257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406ada2-0cfd-47d6-abcc-7dc434b52989 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:13.166926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395289e8-a7f1-4ed6-8f24-6994bbc0d87f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:13.246172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:13.373370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406ada2-0cfd-47d6-abcc-7dc434b52989 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:13.394963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395289e8-a7f1-4ed6-8f24-6994bbc0d87f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:13.439380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:13.446733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35c913-67fb-471d-a7e2-66e27bf86ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:13.451843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35c913-67fb-471d-a7e2-66e27bf86ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:13.451843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35c913-67fb-471d-a7e2-66e27bf86ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:13.451843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35c913-67fb-471d-a7e2-66e27bf86ef4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:13.453311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c48f-6c43-4a57-9b7f-966fa39766b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:13.453963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c48f-6c43-4a57-9b7f-966fa39766b1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:13.454257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c48f-6c43-4a57-9b7f-966fa39766b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:13.454529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c48f-6c43-4a57-9b7f-966fa39766b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:13.462422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35c913-67fb-471d-a7e2-66e27bf86ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-ad35c913-67fb-471d-a7e2-66e27bf86ef4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:13.468524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:13.469412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:13.470190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:13.474468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 82/153] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:13.481035 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35c913-67fb-471d-a7e2-66e27bf86ef4 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:13.483332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5c48f-6c43-4a57-9b7f-966fa39766b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-65c5c48f-6c43-4a57-9b7f-966fa39766b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:13.488576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:13.489613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:13.490465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 72/154] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:13.497703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c48f-6c43-4a57-9b7f-966fa39766b1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:13.627072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c48f-6c43-4a57-9b7f-966fa39766b1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:13.630911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:13.666469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:13.667791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:13.668202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:13.668657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:13.669875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:13.670395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:13.671826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:13.681703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 73/155] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:13.688446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:13.725234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35c913-67fb-471d-a7e2-66e27bf86ef4 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:13.796379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:13.797003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:13.797285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:13.797560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:13.798243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:13.798574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:13.800224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:13.813729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 83/156] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:13.824743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:13.833768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cd37f-97b9-49f9-93d0-ad47a6b31d2a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:13.871140 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23932f65-1e48-44f1-86dc-5375151cf443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:13.871887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23932f65-1e48-44f1-86dc-5375151cf443 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:13.872178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23932f65-1e48-44f1-86dc-5375151cf443 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:13.872484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23932f65-1e48-44f1-86dc-5375151cf443 None admin] PATH_INFO: `/v3/users/6e390f818f21433d9d55c5e4c3ba79e5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:13.873706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23932f65-1e48-44f1-86dc-5375151cf443 None admin] RBAC: Authorizing `identity:get_user(user_id=6e390f818f21433d9d55c5e4c3ba79e5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:13.875257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23932f65-1e48-44f1-86dc-5375151cf443 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:13.877311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 74/157] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:13 2022] GET /identity/v3/users/6e390f818f21433d9d55c5e4c3ba79e5 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:13.882728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23932f65-1e48-44f1-86dc-5375151cf443 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:13.911322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:13.924112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:13.978764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da823e49-5407-4dab-8d87-a60717f101eb None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:13.995822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23932f65-1e48-44f1-86dc-5375151cf443 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:14.033785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91911f-824f-409f-8a11-13e14ea75e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:14.034415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91911f-824f-409f-8a11-13e14ea75e7b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:14.034707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91911f-824f-409f-8a11-13e14ea75e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:14.034993 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91911f-824f-409f-8a11-13e14ea75e7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:14.040338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e193a-ca38-4213-a5ca-89c388bd39ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:14.040984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e193a-ca38-4213-a5ca-89c388bd39ab None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:14.041267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e193a-ca38-4213-a5ca-89c388bd39ab None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:14.041545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e193a-ca38-4213-a5ca-89c388bd39ab None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:14.047798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e193a-ca38-4213-a5ca-89c388bd39ab None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:14.049294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e193a-ca38-4213-a5ca-89c388bd39ab None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:14.051041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 84/158] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:13 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:14.056582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e193a-ca38-4213-a5ca-89c388bd39ab None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:14.118682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91911f-824f-409f-8a11-13e14ea75e7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:14.120783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91911f-824f-409f-8a11-13e14ea75e7b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:14.156715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:14.223977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e193a-ca38-4213-a5ca-89c388bd39ab None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:14.229093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 75/159] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:03:14.235213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91911f-824f-409f-8a11-13e14ea75e7b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:14.249128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:14.268114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cb67b-9d5a-4125-b789-99ff0d0f76b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:14.268811 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cb67b-9d5a-4125-b789-99ff0d0f76b8 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:14.269195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cb67b-9d5a-4125-b789-99ff0d0f76b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:14.269473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cb67b-9d5a-4125-b789-99ff0d0f76b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:14.343687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91911f-824f-409f-8a11-13e14ea75e7b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:14.366994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cb67b-9d5a-4125-b789-99ff0d0f76b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:14.368716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cb67b-9d5a-4125-b789-99ff0d0f76b8 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:14.380170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ed76e-19cf-4fbd-bc76-a855b0d5ed2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:14.380816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed76e-19cf-4fbd-bc76-a855b0d5ed2a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:14.381096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed76e-19cf-4fbd-bc76-a855b0d5ed2a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:14.381369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed76e-19cf-4fbd-bc76-a855b0d5ed2a None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:14.382550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ed76e-19cf-4fbd-bc76-a855b0d5ed2a None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:14.384032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ed76e-19cf-4fbd-bc76-a855b0d5ed2a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:14.389170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 76/160] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:14 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:14.392054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ed76e-19cf-4fbd-bc76-a855b0d5ed2a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:14.402107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:14.527147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 85/161] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:03:14.532659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cb67b-9d5a-4125-b789-99ff0d0f76b8 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:14.550878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ed76e-19cf-4fbd-bc76-a855b0d5ed2a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:14.567445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:14.628621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:14.629632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:14.630110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:14.630575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:14.631552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=5d73bd77f96146698091e497e8e76566, user.id=6e390f818f21433d9d55c5e4c3ba79e5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:14.632058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:14.633699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:14.653600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 77/162] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:14 2022] GET /identity/v3/role_assignments?user.id=6e390f818f21433d9d55c5e4c3ba79e5&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=f72d3388f13149b483355331380f040b => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:14.672488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cb67b-9d5a-4125-b789-99ff0d0f76b8 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:14.722599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df02bf-3b1e-41a1-af5e-ac0a6c263586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:14.723272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df02bf-3b1e-41a1-af5e-ac0a6c263586 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:14.723552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df02bf-3b1e-41a1-af5e-ac0a6c263586 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:14.723825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df02bf-3b1e-41a1-af5e-ac0a6c263586 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:14.724989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df02bf-3b1e-41a1-af5e-ac0a6c263586 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:14.726471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df02bf-3b1e-41a1-af5e-ac0a6c263586 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:14.728289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 86/163] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:14 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:14.734597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:14.861427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:14.921178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e632e-a130-4026-b367-a04b045ab70a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:14.958225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe161803-53c2-441e-8693-8e4e8e343fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:14.959215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe161803-53c2-441e-8693-8e4e8e343fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:14.959751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe161803-53c2-441e-8693-8e4e8e343fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:14.960221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe161803-53c2-441e-8693-8e4e8e343fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:14.961157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe161803-53c2-441e-8693-8e4e8e343fe2 None admin] RBAC: Adding query filter params (role.id=347174cd91d94848aec704f00988e871, scope.project.id=5d73bd77f96146698091e497e8e76566, user.id=18a66efba8ca4c218281db23e0303fe5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:14.961661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe161803-53c2-441e-8693-8e4e8e343fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:14.963326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe161803-53c2-441e-8693-8e4e8e343fe2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:14.970984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 78/164] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:14 2022] GET /identity/v3/role_assignments?user.id=18a66efba8ca4c218281db23e0303fe5&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=347174cd91d94848aec704f00988e871 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:15.024147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5b26f-7648-42dd-a680-a2708ecd2c13 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:15.024463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5b26f-7648-42dd-a680-a2708ecd2c13 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:15.024702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5b26f-7648-42dd-a680-a2708ecd2c13 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:15.026050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 87/165] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:15.032251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886c01-428e-493b-aa82-d5ffa00dc72b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:15.032781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886c01-428e-493b-aa82-d5ffa00dc72b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:15.033252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886c01-428e-493b-aa82-d5ffa00dc72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:15.059997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-63886c01-428e-493b-aa82-d5ffa00dc72b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:15.061536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-63886c01-428e-493b-aa82-d5ffa00dc72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:15.143875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 79/166] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:15.176285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5b26f-7648-42dd-a680-a2708ecd2c13 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:15.308948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:15.324838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5b26f-7648-42dd-a680-a2708ecd2c13 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:15.368427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512c53d-1668-4daa-b49c-3569662ab2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:15.369070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512c53d-1668-4daa-b49c-3569662ab2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:15.369348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512c53d-1668-4daa-b49c-3569662ab2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:15.369617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512c53d-1668-4daa-b49c-3569662ab2e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:15.382122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512c53d-1668-4daa-b49c-3569662ab2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-5512c53d-1668-4daa-b49c-3569662ab2e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:15.387653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:15.388573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:15.389577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:15.392038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 88/167] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:15.399347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63886c01-428e-493b-aa82-d5ffa00dc72b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:15.488638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:15.541329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63886c01-428e-493b-aa82-d5ffa00dc72b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:15.584087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cdbd2-4a51-4a81-950e-e9726d2d6f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:15.584709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdbd2-4a51-4a81-950e-e9726d2d6f01 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:15.584962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdbd2-4a51-4a81-950e-e9726d2d6f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:15.585235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdbd2-4a51-4a81-950e-e9726d2d6f01 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:15.585904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cdbd2-4a51-4a81-950e-e9726d2d6f01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:15.586203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cdbd2-4a51-4a81-950e-e9726d2d6f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:15.587609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cdbd2-4a51-4a81-950e-e9726d2d6f01 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:15.596469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 80/168] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:15.604216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512c53d-1668-4daa-b49c-3569662ab2e5 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:15.614326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302e0e4-85a6-4557-ad24-422aefdabc01 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:15.614611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302e0e4-85a6-4557-ad24-422aefdabc01 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:15.614936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302e0e4-85a6-4557-ad24-422aefdabc01 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:15.616402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 81/169] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:15.622677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af1bc-c6db-45b6-b55a-239a1109edc9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:15.622969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af1bc-c6db-45b6-b55a-239a1109edc9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:15.623303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af1bc-c6db-45b6-b55a-239a1109edc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:15.654575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c47af1bc-c6db-45b6-b55a-239a1109edc9 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:15.655704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c47af1bc-c6db-45b6-b55a-239a1109edc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:15.757524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512c53d-1668-4daa-b49c-3569662ab2e5 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:15.785819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 82/170] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:15.812728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7f5a4-079f-4cc4-8bde-723d41561c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:15.813377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7f5a4-079f-4cc4-8bde-723d41561c60 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:15.813663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7f5a4-079f-4cc4-8bde-723d41561c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:15.813938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7f5a4-079f-4cc4-8bde-723d41561c60 None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:15.815135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7f5a4-079f-4cc4-8bde-723d41561c60 None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:15.816597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7f5a4-079f-4cc4-8bde-723d41561c60 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:15.818189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 89/171] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:15 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:15.823937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47af1bc-c6db-45b6-b55a-239a1109edc9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:15.828153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7f5a4-079f-4cc4-8bde-723d41561c60 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:15.856401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:15.917409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:15.971693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7f5a4-079f-4cc4-8bde-723d41561c60 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:16.004035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47af1bc-c6db-45b6-b55a-239a1109edc9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:16.028485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77143181-1ddd-4243-8c3a-2b70e6a67821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:16.029138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77143181-1ddd-4243-8c3a-2b70e6a67821 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.029448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77143181-1ddd-4243-8c3a-2b70e6a67821 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.029685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77143181-1ddd-4243-8c3a-2b70e6a67821 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.056558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77143181-1ddd-4243-8c3a-2b70e6a67821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-77143181-1ddd-4243-8c3a-2b70e6a67821 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:16.062401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:16.063343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:16.064175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:16.065372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92fad8-5218-48b0-8e48-35387bc6f6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:16.066536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92fad8-5218-48b0-8e48-35387bc6f6bb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.067037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92fad8-5218-48b0-8e48-35387bc6f6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.067603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92fad8-5218-48b0-8e48-35387bc6f6bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.073633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 90/172] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:16.078048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77143181-1ddd-4243-8c3a-2b70e6a67821 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:16.191470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:16.215731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77143181-1ddd-4243-8c3a-2b70e6a67821 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:16.251025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:16.251641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.251925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.252200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.252880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:16.253187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:16.255157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:16.264358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 91/173] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:16.265630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92fad8-5218-48b0-8e48-35387bc6f6bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:16.267147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92fad8-5218-48b0-8e48-35387bc6f6bb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:16.269898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:16.359623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:16.377684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bcb63-9263-4360-9e54-3528101b235e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:16.414279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c27e3-7703-4c07-81a9-b274e02dc88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:16.414906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c27e3-7703-4c07-81a9-b274e02dc88d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.415227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c27e3-7703-4c07-81a9-b274e02dc88d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.415503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c27e3-7703-4c07-81a9-b274e02dc88d None admin] PATH_INFO: `/v3/users/18a66efba8ca4c218281db23e0303fe5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.420710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c27e3-7703-4c07-81a9-b274e02dc88d None admin] RBAC: Authorizing `identity:get_user(user_id=18a66efba8ca4c218281db23e0303fe5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:16.422179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c27e3-7703-4c07-81a9-b274e02dc88d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:16.424118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 92/174] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:16 2022] GET /identity/v3/users/18a66efba8ca4c218281db23e0303fe5 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:16.431331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c27e3-7703-4c07-81a9-b274e02dc88d None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:16.452375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 83/175] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:16.453985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:16.458741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92fad8-5218-48b0-8e48-35387bc6f6bb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:16.573451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c27e3-7703-4c07-81a9-b274e02dc88d None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:16.578194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92fad8-5218-48b0-8e48-35387bc6f6bb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:16.610137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17912ba6-8b2c-4d02-afae-c6ef7080620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:16.610780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912ba6-8b2c-4d02-afae-c6ef7080620a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.611027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912ba6-8b2c-4d02-afae-c6ef7080620a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.611314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912ba6-8b2c-4d02-afae-c6ef7080620a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.624817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9a0f7-69e6-4907-8309-60b4e7d14f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:16.625512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9a0f7-69e6-4907-8309-60b4e7d14f66 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.625791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9a0f7-69e6-4907-8309-60b4e7d14f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.626063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9a0f7-69e6-4907-8309-60b4e7d14f66 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.631715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9a0f7-69e6-4907-8309-60b4e7d14f66 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:16.633245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9a0f7-69e6-4907-8309-60b4e7d14f66 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:16.635293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 84/176] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:16 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:16.636893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66584f21-54ec-4cfb-96bb-e2ac6ac9876c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.637176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66584f21-54ec-4cfb-96bb-e2ac6ac9876c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.637450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66584f21-54ec-4cfb-96bb-e2ac6ac9876c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.638673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 85/177] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:16.642836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66584f21-54ec-4cfb-96bb-e2ac6ac9876c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:16.677381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:16.686138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17912ba6-8b2c-4d02-afae-c6ef7080620a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:16.687889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17912ba6-8b2c-4d02-afae-c6ef7080620a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:16.698312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:16.799870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 93/178] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:16.804197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d62ea-ea1d-402e-b526-6e8eb24d637e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.804197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d62ea-ea1d-402e-b526-6e8eb24d637e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.804197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d62ea-ea1d-402e-b526-6e8eb24d637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.814015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66584f21-54ec-4cfb-96bb-e2ac6ac9876c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:16.821095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-dc0d62ea-ea1d-402e-b526-6e8eb24d637e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:16.822642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-dc0d62ea-ea1d-402e-b526-6e8eb24d637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:16.856707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e5803-d23f-490a-9352-527d5018bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:16.857880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e5803-d23f-490a-9352-527d5018bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.858416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e5803-d23f-490a-9352-527d5018bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.858855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e5803-d23f-490a-9352-527d5018bbd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.859689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e5803-d23f-490a-9352-527d5018bbd0 None admin] RBAC: Adding query filter params (role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=5d73bd77f96146698091e497e8e76566, user.id=18a66efba8ca4c218281db23e0303fe5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:16.860070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e5803-d23f-490a-9352-527d5018bbd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:16.861650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e5803-d23f-490a-9352-527d5018bbd0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:16.869284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 86/179] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:16 2022] GET /identity/v3/role_assignments?user.id=18a66efba8ca4c218281db23e0303fe5&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:16.871585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eddfb-c127-40ec-add8-b5d9e1096469 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.871960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eddfb-c127-40ec-add8-b5d9e1096469 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.872309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eddfb-c127-40ec-add8-b5d9e1096469 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.874382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 87/180] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:16.878525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147eddfb-c127-40ec-add8-b5d9e1096469 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:16.911040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 94/181] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:16.915200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6aed05-3a00-46a3-ba41-f48fa611db8f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:16.915505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6aed05-3a00-46a3-ba41-f48fa611db8f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:16.915778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6aed05-3a00-46a3-ba41-f48fa611db8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:16.957975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-cc6aed05-3a00-46a3-ba41-f48fa611db8f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:16.959252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cc6aed05-3a00-46a3-ba41-f48fa611db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:17.030493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:17.053513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:17.069615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147eddfb-c127-40ec-add8-b5d9e1096469 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.091385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 95/182] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:17.092126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6aed05-3a00-46a3-ba41-f48fa611db8f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.123848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3823d8-8818-40fb-952a-04dd58c90c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.124445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3823d8-8818-40fb-952a-04dd58c90c43 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.125558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3823d8-8818-40fb-952a-04dd58c90c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.125558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3823d8-8818-40fb-952a-04dd58c90c43 None admin] PATH_INFO: `/v3/projects/1343820eab844e92a91697087f8c9830` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.126707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3823d8-8818-40fb-952a-04dd58c90c43 None admin] RBAC: Authorizing `identity:get_project(project_id=1343820eab844e92a91697087f8c9830)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:17.128434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3823d8-8818-40fb-952a-04dd58c90c43 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:17.130346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 88/183] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:16 2022] GET /identity/v3/projects/1343820eab844e92a91697087f8c9830 => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:17.135339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f0be-b582-46d4-95b5-d3a7ddfa146e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.135339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f0be-b582-46d4-95b5-d3a7ddfa146e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.135339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1f0be-b582-46d4-95b5-d3a7ddfa146e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.135339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 89/184] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:17.135339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1f0be-b582-46d4-95b5-d3a7ddfa146e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.214203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6aed05-3a00-46a3-ba41-f48fa611db8f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.249645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e2e62-df7a-4623-9feb-b4183649ad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.250317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e2e62-df7a-4623-9feb-b4183649ad8c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.250606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e2e62-df7a-4623-9feb-b4183649ad8c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.250864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e2e62-df7a-4623-9feb-b4183649ad8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.257973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1f0be-b582-46d4-95b5-d3a7ddfa146e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.261555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e2e62-df7a-4623-9feb-b4183649ad8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-2b7e2e62-df7a-4623-9feb-b4183649ad8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:17.266291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:17.267485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 96/185] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:17.273493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e2e62-df7a-4623-9feb-b4183649ad8c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.302719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:17.305720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47841d-021e-470b-9e54-f17e2fd0d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.306541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47841d-021e-470b-9e54-f17e2fd0d593 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.306825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47841d-021e-470b-9e54-f17e2fd0d593 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.307105 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47841d-021e-470b-9e54-f17e2fd0d593 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.318371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47841d-021e-470b-9e54-f17e2fd0d593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ac47841d-021e-470b-9e54-f17e2fd0d593 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:17.323408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:17.324504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:17.325888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 90/186] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:17.328310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63841-6bbf-418e-9fde-fa910488eb48 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.328596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63841-6bbf-418e-9fde-fa910488eb48 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.328887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63841-6bbf-418e-9fde-fa910488eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.346890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cbf63841-6bbf-418e-9fde-fa910488eb48 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:17.348121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cbf63841-6bbf-418e-9fde-fa910488eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:17.355994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:17.383807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e2e62-df7a-4623-9feb-b4183649ad8c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.415018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 91/187] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:17.415936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf63841-6bbf-418e-9fde-fa910488eb48 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.421455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3ed7b-75c2-4eef-8c5c-bf46437a9112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.422058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ed7b-75c2-4eef-8c5c-bf46437a9112 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.422336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ed7b-75c2-4eef-8c5c-bf46437a9112 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.422660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ed7b-75c2-4eef-8c5c-bf46437a9112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.423424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3ed7b-75c2-4eef-8c5c-bf46437a9112 None admin] RBAC: Adding query filter params (role.id=347174cd91d94848aec704f00988e871, scope.project.id=1343820eab844e92a91697087f8c9830, user.id=18a66efba8ca4c218281db23e0303fe5) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:17.423735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3ed7b-75c2-4eef-8c5c-bf46437a9112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:17.425210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3ed7b-75c2-4eef-8c5c-bf46437a9112 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:17.435958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 97/188] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/role_assignments?user.id=18a66efba8ca4c218281db23e0303fe5&scope.project.id=1343820eab844e92a91697087f8c9830&role.id=347174cd91d94848aec704f00988e871 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:17.435958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f78e76-4b0b-474b-8831-66b109563370 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.435958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f78e76-4b0b-474b-8831-66b109563370 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.435958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f78e76-4b0b-474b-8831-66b109563370 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.435958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 98/189] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:17.435958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f78e76-4b0b-474b-8831-66b109563370 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.540900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf63841-6bbf-418e-9fde-fa910488eb48 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.551142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:17.561819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f78e76-4b0b-474b-8831-66b109563370 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.580577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.581197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.581480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.582264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.582953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:17.583328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:17.584853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:17.597032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 92/190] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:17.598185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.602628 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:17.607524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838297d0-0edc-4c44-8e88-9e3b7147cce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.607822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838297d0-0edc-4c44-8e88-9e3b7147cce4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.608398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838297d0-0edc-4c44-8e88-9e3b7147cce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.608398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838297d0-0edc-4c44-8e88-9e3b7147cce4 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.609044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838297d0-0edc-4c44-8e88-9e3b7147cce4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:17.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838297d0-0edc-4c44-8e88-9e3b7147cce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:17.611017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838297d0-0edc-4c44-8e88-9e3b7147cce4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:17.623051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 99/191] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:17.623051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3f9b-f8b7-4164-ac97-91f70bde64e4 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.623051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3f9b-f8b7-4164-ac97-91f70bde64e4 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.623051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3f9b-f8b7-4164-ac97-91f70bde64e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.642891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f1ba3f9b-f8b7-4164-ac97-91f70bde64e4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:17.644127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f1ba3f9b-f8b7-4164-ac97-91f70bde64e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:17.710762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee1c04-fb2b-4ee1-be2e-22d94705102e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.717218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 100/192] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:17.717893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba3f9b-f8b7-4164-ac97-91f70bde64e4 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.747187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff1da-01b7-4216-9ab4-c820aea67f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.747820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff1da-01b7-4216-9ab4-c820aea67f85 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.748097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff1da-01b7-4216-9ab4-c820aea67f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.748374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff1da-01b7-4216-9ab4-c820aea67f85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.758232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edff1da-01b7-4216-9ab4-c820aea67f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7edff1da-01b7-4216-9ab4-c820aea67f85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:17.763522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:17.764599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 93/193] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:17.765425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff1da-01b7-4216-9ab4-c820aea67f85 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.816985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:17.826847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:17.832782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba3f9b-f8b7-4164-ac97-91f70bde64e4 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.863188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff1da-01b7-4216-9ab4-c820aea67f85 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:17.866047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1c45-0a2c-465a-9501-c00c3a394f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.866673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1c45-0a2c-465a-9501-c00c3a394f40 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.866949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1c45-0a2c-465a-9501-c00c3a394f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.867245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1c45-0a2c-465a-9501-c00c3a394f40 None admin] PATH_INFO: `/v3/users/1877997db3804c1c892eee99cebba58a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.881408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a1c45-0a2c-465a-9501-c00c3a394f40 None admin] RBAC: Authorizing `identity:get_user(user_id=1877997db3804c1c892eee99cebba58a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:17.882859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a1c45-0a2c-465a-9501-c00c3a394f40 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:17.884901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 101/194] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/users/1877997db3804c1c892eee99cebba58a => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:17.886410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1c45-0a2c-465a-9501-c00c3a394f40 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.898038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86465ddd-dc60-4981-b2ba-2670b9ccbac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:17.898740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465ddd-dc60-4981-b2ba-2670b9ccbac5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:17.899046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465ddd-dc60-4981-b2ba-2670b9ccbac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:17.899323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86465ddd-dc60-4981-b2ba-2670b9ccbac5 None admin] PATH_INFO: `/v3/users/1877997db3804c1c892eee99cebba58a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:17.900442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86465ddd-dc60-4981-b2ba-2670b9ccbac5 None admin] RBAC: Authorizing `identity:get_user(user_id=1877997db3804c1c892eee99cebba58a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:17.901926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86465ddd-dc60-4981-b2ba-2670b9ccbac5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:17.903517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 94/195] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/users/1877997db3804c1c892eee99cebba58a => generated 283 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:17.904269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86465ddd-dc60-4981-b2ba-2670b9ccbac5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:17.988206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1c45-0a2c-465a-9501-c00c3a394f40 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.001560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86465ddd-dc60-4981-b2ba-2670b9ccbac5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.022067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d631b4-14fd-4d87-a5e5-a61d9fcf327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.022689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d631b4-14fd-4d87-a5e5-a61d9fcf327c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.022970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d631b4-14fd-4d87-a5e5-a61d9fcf327c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.023284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d631b4-14fd-4d87-a5e5-a61d9fcf327c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.024391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d631b4-14fd-4d87-a5e5-a61d9fcf327c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.025848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d631b4-14fd-4d87-a5e5-a61d9fcf327c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.027614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 102/196] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:18.032321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d631b4-14fd-4d87-a5e5-a61d9fcf327c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:18.032433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.035327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.036204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.036474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.037139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:18.037440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.038790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.042104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.047083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 95/197] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:18.047876 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:18.130154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d631b4-14fd-4d87-a5e5-a61d9fcf327c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.148532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c40e1-1eaf-4855-91dc-47b70623832e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.163816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a650b-af2a-40c1-92ca-0f786a1954f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.164407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a650b-af2a-40c1-92ca-0f786a1954f6 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.164678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a650b-af2a-40c1-92ca-0f786a1954f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.164942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a650b-af2a-40c1-92ca-0f786a1954f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.181342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7474955-1c50-4a0f-9e5a-df6b9c755b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.181938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474955-1c50-4a0f-9e5a-df6b9c755b47 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.182215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474955-1c50-4a0f-9e5a-df6b9c755b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.183051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7474955-1c50-4a0f-9e5a-df6b9c755b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.229934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a650b-af2a-40c1-92ca-0f786a1954f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.231423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a650b-af2a-40c1-92ca-0f786a1954f6 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.242284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.242868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.250173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7474955-1c50-4a0f-9e5a-df6b9c755b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.251627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7474955-1c50-4a0f-9e5a-df6b9c755b47 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.328668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 103/198] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:18.333526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a650b-af2a-40c1-92ca-0f786a1954f6 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:18.355981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 96/199] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:18.359927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7474955-1c50-4a0f-9e5a-df6b9c755b47 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:18.426164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a650b-af2a-40c1-92ca-0f786a1954f6 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.436541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.450444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.456873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7474955-1c50-4a0f-9e5a-df6b9c755b47 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.461760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f29b4-bda1-4929-9d20-cbf19f3773e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.462412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f29b4-bda1-4929-9d20-cbf19f3773e7 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.462692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f29b4-bda1-4929-9d20-cbf19f3773e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.463190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f29b4-bda1-4929-9d20-cbf19f3773e7 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.464521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f29b4-bda1-4929-9d20-cbf19f3773e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.466329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f29b4-bda1-4929-9d20-cbf19f3773e7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.491314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4d5aa-3fd1-4827-bfd3-7f9f5e1bfcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.491940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4d5aa-3fd1-4827-bfd3-7f9f5e1bfcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.492220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4d5aa-3fd1-4827-bfd3-7f9f5e1bfcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.492504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4d5aa-3fd1-4827-bfd3-7f9f5e1bfcf1 None admin] PATH_INFO: `/v3/users/01a2cd5391ee4c44923895da0d75c715` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.493618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4d5aa-3fd1-4827-bfd3-7f9f5e1bfcf1 None admin] RBAC: Authorizing `identity:get_user(user_id=01a2cd5391ee4c44923895da0d75c715)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.495113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4d5aa-3fd1-4827-bfd3-7f9f5e1bfcf1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.496677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 97/200] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:18 2022] GET /identity/v3/users/01a2cd5391ee4c44923895da0d75c715 => generated 289 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:18.497427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4d5aa-3fd1-4827-bfd3-7f9f5e1bfcf1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:18.596068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4d5aa-3fd1-4827-bfd3-7f9f5e1bfcf1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.615433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 104/201] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:18 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:18.616821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f29b4-bda1-4929-9d20-cbf19f3773e7 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:18.642432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911c40-0bc5-4c42-bcc0-53fea7e353c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.643177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911c40-0bc5-4c42-bcc0-53fea7e353c1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.643450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911c40-0bc5-4c42-bcc0-53fea7e353c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.643723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911c40-0bc5-4c42-bcc0-53fea7e353c1 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.645236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31911c40-0bc5-4c42-bcc0-53fea7e353c1 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.646752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31911c40-0bc5-4c42-bcc0-53fea7e353c1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.654516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 98/202] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:18 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:18.654516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911c40-0bc5-4c42-bcc0-53fea7e353c1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:18.690974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.741131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f29b4-bda1-4929-9d20-cbf19f3773e7 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.771292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911c40-0bc5-4c42-bcc0-53fea7e353c1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.783243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255d11-fc1c-4923-91b9-f8510c70d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.785089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255d11-fc1c-4923-91b9-f8510c70d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.785089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255d11-fc1c-4923-91b9-f8510c70d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.785268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255d11-fc1c-4923-91b9-f8510c70d3f1 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.786355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4255d11-fc1c-4923-91b9-f8510c70d3f1 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.787855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4255d11-fc1c-4923-91b9-f8510c70d3f1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.789488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 105/203] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:18 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:18.792002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255d11-fc1c-4923-91b9-f8510c70d3f1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:18.810133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96228111-1955-489b-852f-d00d1f6a9312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.810744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228111-1955-489b-852f-d00d1f6a9312 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.811041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228111-1955-489b-852f-d00d1f6a9312 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.811317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228111-1955-489b-852f-d00d1f6a9312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.830284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.903450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96228111-1955-489b-852f-d00d1f6a9312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.904930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96228111-1955-489b-852f-d00d1f6a9312 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.920020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255d11-fc1c-4923-91b9-f8510c70d3f1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:18.938626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:18.966755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:18.967451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:18.967730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:18.967999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:18.968829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=456db541138a40ea9d931140527e9c4a, user.id=1877997db3804c1c892eee99cebba58a) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:18.969133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:18.970594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:18.977503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 106/204] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:18 2022] GET /identity/v3/role_assignments?user.id=1877997db3804c1c892eee99cebba58a&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=f72d3388f13149b483355331380f040b => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:18.978594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:19.078620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 99/205] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:19.084500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96228111-1955-489b-852f-d00d1f6a9312 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:19.129939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:19.136833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df4b89-65cd-49ca-8a36-3f990cbbea09 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:19.227121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:19.227121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:19.227121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:19.227121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:19.227606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] RBAC: Adding query filter params (role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=456db541138a40ea9d931140527e9c4a, user.id=1877997db3804c1c892eee99cebba58a) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:19.227606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:19.227606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:19.227606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 107/206] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:18 2022] GET /identity/v3/role_assignments?user.id=1877997db3804c1c892eee99cebba58a&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:19.307564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96228111-1955-489b-852f-d00d1f6a9312 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:19.348439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:19.360959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef96636-5fc9-4484-9889-7d85d10077f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:19.366302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef96636-5fc9-4484-9889-7d85d10077f4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:19.366302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef96636-5fc9-4484-9889-7d85d10077f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:19.366302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef96636-5fc9-4484-9889-7d85d10077f4 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:19.366302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef96636-5fc9-4484-9889-7d85d10077f4 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:19.366302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef96636-5fc9-4484-9889-7d85d10077f4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:19.368013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 100/207] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:19 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:19.373563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:19.506581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20168e6-3574-4778-ac57-2eb0adcce022 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:19.554517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82991ee-380e-4f7f-a9cd-2899fa7d645d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:19.555157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82991ee-380e-4f7f-a9cd-2899fa7d645d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:19.555440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82991ee-380e-4f7f-a9cd-2899fa7d645d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:19.555712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82991ee-380e-4f7f-a9cd-2899fa7d645d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:19.556485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82991ee-380e-4f7f-a9cd-2899fa7d645d None admin] RBAC: Adding query filter params (role.id=a5998ecc28d44e0c9deb0653af1c8eaf, scope.project.id=5d73bd77f96146698091e497e8e76566, user.id=01a2cd5391ee4c44923895da0d75c715) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:19.556792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82991ee-380e-4f7f-a9cd-2899fa7d645d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:19.559204 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82991ee-380e-4f7f-a9cd-2899fa7d645d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:19.565779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 108/208] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:19 2022] GET /identity/v3/role_assignments?user.id=01a2cd5391ee4c44923895da0d75c715&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=a5998ecc28d44e0c9deb0653af1c8eaf => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:20.894121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be863c2-15b1-487a-ac8d-8650958323f9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:20.894646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be863c2-15b1-487a-ac8d-8650958323f9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:20.894646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be863c2-15b1-487a-ac8d-8650958323f9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:20.896046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 101/209] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:20.901964 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd6186-5c57-4893-88c4-0e119b965186 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:20.902384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd6186-5c57-4893-88c4-0e119b965186 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:20.902482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd6186-5c57-4893-88c4-0e119b965186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:20.921195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1fdd6186-5c57-4893-88c4-0e119b965186 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:20.922385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1fdd6186-5c57-4893-88c4-0e119b965186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:20.944763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:20.999124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 109/210] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:21.019188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be863c2-15b1-487a-ac8d-8650958323f9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.026908 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5acb07-c5a1-4711-857e-b5b108e4c1c0 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.027271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5acb07-c5a1-4711-857e-b5b108e4c1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.027476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5acb07-c5a1-4711-857e-b5b108e4c1c0 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.028819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 110/211] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:21.033848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6648028-2cc1-40a5-92a1-371bb862deb8 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.034126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6648028-2cc1-40a5-92a1-371bb862deb8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.034424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6648028-2cc1-40a5-92a1-371bb862deb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.053480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-b6648028-2cc1-40a5-92a1-371bb862deb8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:21.054756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b6648028-2cc1-40a5-92a1-371bb862deb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:21.136685 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be863c2-15b1-487a-ac8d-8650958323f9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:21.148726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 111/212] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:21.150616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437a59a-b536-4c66-9ced-b04725a28142 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.151854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437a59a-b536-4c66-9ced-b04725a28142 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.152319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437a59a-b536-4c66-9ced-b04725a28142 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.154760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 112/213] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:21.160311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde4be-e817-4b72-a80d-f9958a5d3f59 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.160601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde4be-e817-4b72-a80d-f9958a5d3f59 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.160882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde4be-e817-4b72-a80d-f9958a5d3f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.179453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-78bde4be-e817-4b72-a80d-f9958a5d3f59 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:21.179841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94d3b-e836-45fa-9c3f-a697029a66b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:21.180475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94d3b-e836-45fa-9c3f-a697029a66b7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.180640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-78bde4be-e817-4b72-a80d-f9958a5d3f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:21.180795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94d3b-e836-45fa-9c3f-a697029a66b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.181035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94d3b-e836-45fa-9c3f-a697029a66b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.191576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a94d3b-e836-45fa-9c3f-a697029a66b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-93a94d3b-e836-45fa-9c3f-a697029a66b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:21.196732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:21.197669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:21.198837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 102/214] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:21.203477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94d3b-e836-45fa-9c3f-a697029a66b7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.254875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 113/215] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:21.263280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bde4be-e817-4b72-a80d-f9958a5d3f59 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.273356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:21.278085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:21.324996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94d3b-e836-45fa-9c3f-a697029a66b7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:21.363659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252e06-4c3e-434c-8d30-63b6e3152d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:21.364297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252e06-4c3e-434c-8d30-63b6e3152d3b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.364637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252e06-4c3e-434c-8d30-63b6e3152d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.364920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252e06-4c3e-434c-8d30-63b6e3152d3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.366037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252e06-4c3e-434c-8d30-63b6e3152d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:21.367522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252e06-4c3e-434c-8d30-63b6e3152d3b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:21.369087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 103/216] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:21.371198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252e06-4c3e-434c-8d30-63b6e3152d3b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.375988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bde4be-e817-4b72-a80d-f9958a5d3f59 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:21.412046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:21.412622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.412853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.413128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.414446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:21.414446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:21.415517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:21.429805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 114/217] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:21.430468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.481512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252e06-4c3e-434c-8d30-63b6e3152d3b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:21.508481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:21.516752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4679ad-cd6a-44cb-8b2f-8cfbe4e028fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:21.517383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4679ad-cd6a-44cb-8b2f-8cfbe4e028fc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.517660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4679ad-cd6a-44cb-8b2f-8cfbe4e028fc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.517769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:21.517931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4679ad-cd6a-44cb-8b2f-8cfbe4e028fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.528489 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4679ad-cd6a-44cb-8b2f-8cfbe4e028fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1e4679ad-cd6a-44cb-8b2f-8cfbe4e028fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:21.534074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:21.535143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:21.536035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 104/218] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:21.536770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4679ad-cd6a-44cb-8b2f-8cfbe4e028fc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.545449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f434f-c18d-46bb-90f9-8a66d2b5ed49 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:21.579262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d682801-d44a-4136-8af0-47c84343cfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:21.579853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d682801-d44a-4136-8af0-47c84343cfca None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.580129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d682801-d44a-4136-8af0-47c84343cfca None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.580406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d682801-d44a-4136-8af0-47c84343cfca None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.581037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d682801-d44a-4136-8af0-47c84343cfca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:21.582479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d682801-d44a-4136-8af0-47c84343cfca None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:21.638309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4679ad-cd6a-44cb-8b2f-8cfbe4e028fc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:21.655174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 115/219] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:21 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:21.656650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4c2e1-f1b8-4943-9855-d85485e7fce2 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.656939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4c2e1-f1b8-4943-9855-d85485e7fce2 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.657217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4c2e1-f1b8-4943-9855-d85485e7fce2 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.658479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 116/220] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:21.662858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4c2e1-f1b8-4943-9855-d85485e7fce2 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.688747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceafe11-1b03-4467-8b2e-724ea130abe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:21.689587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafe11-1b03-4467-8b2e-724ea130abe2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.689922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafe11-1b03-4467-8b2e-724ea130abe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.690568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafe11-1b03-4467-8b2e-724ea130abe2 None admin] PATH_INFO: `/v3/users/2acaaf265ad04006b03be9d44d9ad025` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.707094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceafe11-1b03-4467-8b2e-724ea130abe2 None admin] RBAC: Authorizing `identity:get_user(user_id=2acaaf265ad04006b03be9d44d9ad025)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:21.708614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceafe11-1b03-4467-8b2e-724ea130abe2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:21.710351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 105/221] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:21 2022] GET /identity/v3/users/2acaaf265ad04006b03be9d44d9ad025 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:21.711690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bab68a-ea85-4e34-95eb-063008ca16d2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.711976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bab68a-ea85-4e34-95eb-063008ca16d2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.712249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bab68a-ea85-4e34-95eb-063008ca16d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.731320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b7bab68a-ea85-4e34-95eb-063008ca16d2 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:21.732581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b7bab68a-ea85-4e34-95eb-063008ca16d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:21.779130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:21.794005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4c2e1-f1b8-4943-9855-d85485e7fce2 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:21.815024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 106/222] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:21.815024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bab68a-ea85-4e34-95eb-063008ca16d2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.847456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:21.848372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.848813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.849247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.849979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:21.850284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:21.851717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:21.853304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:21.863592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 117/223] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:21.867032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:21.924085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bab68a-ea85-4e34-95eb-063008ca16d2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:21.977147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035b0c3-4a34-43fe-9800-5df58641567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:21.977764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035b0c3-4a34-43fe-9800-5df58641567b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:21.978041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035b0c3-4a34-43fe-9800-5df58641567b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:21.981137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035b0c3-4a34-43fe-9800-5df58641567b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:21.996849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62152fea-aa91-4188-8779-a5e7dff2324f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.037405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a77841-8798-4e52-af75-e4cec89dd215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.038084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a77841-8798-4e52-af75-e4cec89dd215 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.038361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a77841-8798-4e52-af75-e4cec89dd215 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.038632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a77841-8798-4e52-af75-e4cec89dd215 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.041464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:22.052351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a77841-8798-4e52-af75-e4cec89dd215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-28a77841-8798-4e52-af75-e4cec89dd215 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:22.057303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:22.058456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 118/224] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:22.059308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a77841-8798-4e52-af75-e4cec89dd215 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:22.076110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035b0c3-4a34-43fe-9800-5df58641567b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.078547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035b0c3-4a34-43fe-9800-5df58641567b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.132084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:22.191122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a77841-8798-4e52-af75-e4cec89dd215 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.203363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 107/225] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:22.204591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035b0c3-4a34-43fe-9800-5df58641567b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:22.236433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5f24-d626-477c-afa8-39c66eaa5617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.238164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5f24-d626-477c-afa8-39c66eaa5617 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.238666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5f24-d626-477c-afa8-39c66eaa5617 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.239114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5f24-d626-477c-afa8-39c66eaa5617 None admin] PATH_INFO: `/v3/users/1877997db3804c1c892eee99cebba58a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.240661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a5f24-d626-477c-afa8-39c66eaa5617 None admin] RBAC: Authorizing `identity:get_user(user_id=1877997db3804c1c892eee99cebba58a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.243170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a5f24-d626-477c-afa8-39c66eaa5617 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.245380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 119/226] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:22 2022] GET /identity/v3/users/1877997db3804c1c892eee99cebba58a => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:22.246337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5f24-d626-477c-afa8-39c66eaa5617 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:22.290777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:22.330379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035b0c3-4a34-43fe-9800-5df58641567b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.374750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.375662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.376089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.376455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5f24-d626-477c-afa8-39c66eaa5617 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.376686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.377644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:22.378102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.380564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.390402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 108/227] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:22.391731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:22.406736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:22.421523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc053fd-be30-4ece-868e-4a48ecd8db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.422448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc053fd-be30-4ece-868e-4a48ecd8db8c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.422884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc053fd-be30-4ece-868e-4a48ecd8db8c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.423344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc053fd-be30-4ece-868e-4a48ecd8db8c None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.424830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc053fd-be30-4ece-868e-4a48ecd8db8c None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.426699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc053fd-be30-4ece-868e-4a48ecd8db8c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.428814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 120/228] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:22 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:22.429807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc053fd-be30-4ece-868e-4a48ecd8db8c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:22.502700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc2b17-03bd-46ff-82bf-74d7e676c771 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.538501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306f9ad-22fd-4bf9-ba59-f3cb5bbe0197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.539222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f9ad-22fd-4bf9-ba59-f3cb5bbe0197 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.539573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f9ad-22fd-4bf9-ba59-f3cb5bbe0197 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.539922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f9ad-22fd-4bf9-ba59-f3cb5bbe0197 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.541122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306f9ad-22fd-4bf9-ba59-f3cb5bbe0197 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.542765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306f9ad-22fd-4bf9-ba59-f3cb5bbe0197 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.543809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc053fd-be30-4ece-868e-4a48ecd8db8c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.549539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 109/229] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:22 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:22.550523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306f9ad-22fd-4bf9-ba59-f3cb5bbe0197 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:22.554295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:22.583234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee17f6f-637e-4771-8cd8-34a4acab23b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.584160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17f6f-637e-4771-8cd8-34a4acab23b3 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.584649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17f6f-637e-4771-8cd8-34a4acab23b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.585134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17f6f-637e-4771-8cd8-34a4acab23b3 None admin] PATH_INFO: `/v3/users/1877997db3804c1c892eee99cebba58a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.586652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee17f6f-637e-4771-8cd8-34a4acab23b3 None admin] RBAC: Authorizing `identity:get_user(user_id=1877997db3804c1c892eee99cebba58a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.589884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee17f6f-637e-4771-8cd8-34a4acab23b3 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.591847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 121/230] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:22 2022] GET /identity/v3/users/1877997db3804c1c892eee99cebba58a => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:22.592809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee17f6f-637e-4771-8cd8-34a4acab23b3 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:22.656676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306f9ad-22fd-4bf9-ba59-f3cb5bbe0197 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.660145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:22.691075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.691827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.692036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.692278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.693003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] RBAC: Adding query filter params (role.id=347174cd91d94848aec704f00988e871, scope.project.id=456db541138a40ea9d931140527e9c4a, user.id=2acaaf265ad04006b03be9d44d9ad025) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:22.693308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.694743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.700528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 110/231] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:22 2022] GET /identity/v3/role_assignments?user.id=2acaaf265ad04006b03be9d44d9ad025&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=347174cd91d94848aec704f00988e871 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:22.701120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:22.709621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee17f6f-637e-4771-8cd8-34a4acab23b3 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.751054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd6be3-aa2a-4711-a456-16fd2a5ada6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.752229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd6be3-aa2a-4711-a456-16fd2a5ada6a None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.752848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd6be3-aa2a-4711-a456-16fd2a5ada6a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.753429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd6be3-aa2a-4711-a456-16fd2a5ada6a None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a/users/1877997db3804c1c892eee99cebba58a/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.755890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd6be3-aa2a-4711-a456-16fd2a5ada6a None admin] RBAC: Authorizing `identity:create_grant(project_id=456db541138a40ea9d931140527e9c4a, user_id=1877997db3804c1c892eee99cebba58a, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.757837 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd6be3-aa2a-4711-a456-16fd2a5ada6a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.775021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:22.818632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d9ce3-bcb6-40bb-8ea7-e313bf5a256d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:22.849468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 122/232] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:22 2022] PUT /identity/v3/projects/456db541138a40ea9d931140527e9c4a/users/1877997db3804c1c892eee99cebba58a/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:22.865828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d35bd-3604-4d1c-9f19-86a15b0d8662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:22.866924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d35bd-3604-4d1c-9f19-86a15b0d8662 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:22.867441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d35bd-3604-4d1c-9f19-86a15b0d8662 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:22.867944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d35bd-3604-4d1c-9f19-86a15b0d8662 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:22.869626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d35bd-3604-4d1c-9f19-86a15b0d8662 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:22.872156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d35bd-3604-4d1c-9f19-86a15b0d8662 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:22.874385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 111/233] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:22 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:22.880898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd6be3-aa2a-4711-a456-16fd2a5ada6a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:23.012012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd6be3-aa2a-4711-a456-16fd2a5ada6a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:23.047474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:23.061793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54de7e3-5220-4d08-bd6d-14bb438e64e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:23.062531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7e3-5220-4d08-bd6d-14bb438e64e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:23.062812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7e3-5220-4d08-bd6d-14bb438e64e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:23.063103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54de7e3-5220-4d08-bd6d-14bb438e64e9 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a/users/1877997db3804c1c892eee99cebba58a/roles/9475450bfdd04912abb88ccf9973f9d8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:23.065435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54de7e3-5220-4d08-bd6d-14bb438e64e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=456db541138a40ea9d931140527e9c4a, user_id=1877997db3804c1c892eee99cebba58a, role_id=9475450bfdd04912abb88ccf9973f9d8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:23.066949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54de7e3-5220-4d08-bd6d-14bb438e64e9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:23.140659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 123/234] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:22 2022] PUT /identity/v3/projects/456db541138a40ea9d931140527e9c4a/users/1877997db3804c1c892eee99cebba58a/roles/9475450bfdd04912abb88ccf9973f9d8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:24.039968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310ff2-87a9-4377-8161-b7532870d608 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.040483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310ff2-87a9-4377-8161-b7532870d608 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.040483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310ff2-87a9-4377-8161-b7532870d608 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.042733 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 112/235] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:24.048054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f5d2-579e-4bce-a4b9-2c87f8d7b5b5 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.048459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f5d2-579e-4bce-a4b9-2c87f8d7b5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.048663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f5d2-579e-4bce-a4b9-2c87f8d7b5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.075996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3616f5d2-579e-4bce-a4b9-2c87f8d7b5b5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:24.077322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3616f5d2-579e-4bce-a4b9-2c87f8d7b5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:24.163306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 124/236] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:24.168470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1429e3-8f6a-4a45-8f0c-73e108f379df None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.168760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1429e3-8f6a-4a45-8f0c-73e108f379df None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.169027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1429e3-8f6a-4a45-8f0c-73e108f379df None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.170597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 113/237] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:24.175896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109382fb-61ca-4d3e-93a9-8744acea9d24 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.176187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109382fb-61ca-4d3e-93a9-8744acea9d24 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.176464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109382fb-61ca-4d3e-93a9-8744acea9d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.202809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1429e3-8f6a-4a45-8f0c-73e108f379df None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:24.204196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-109382fb-61ca-4d3e-93a9-8744acea9d24 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:24.205418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-109382fb-61ca-4d3e-93a9-8744acea9d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:24.324706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 125/238] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:24.349941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1429e3-8f6a-4a45-8f0c-73e108f379df None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:24.371105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109382fb-61ca-4d3e-93a9-8744acea9d24 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:24.399022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37ac73-ea46-4a6f-8463-4417bc19d69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:24.399656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37ac73-ea46-4a6f-8463-4417bc19d69f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.400721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37ac73-ea46-4a6f-8463-4417bc19d69f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.401617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37ac73-ea46-4a6f-8463-4417bc19d69f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.409150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:24.423535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37ac73-ea46-4a6f-8463-4417bc19d69f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5c37ac73-ea46-4a6f-8463-4417bc19d69f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:24.431196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:24.434767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:24.435610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 114/239] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:24.440210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37ac73-ea46-4a6f-8463-4417bc19d69f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:24.456844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:24.561276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109382fb-61ca-4d3e-93a9-8744acea9d24 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:24.623300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec470-7ad2-4a2d-ae03-b3d431d6d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:24.623941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec470-7ad2-4a2d-ae03-b3d431d6d633 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.624217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec470-7ad2-4a2d-ae03-b3d431d6d633 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.624489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec470-7ad2-4a2d-ae03-b3d431d6d633 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.626759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37ac73-ea46-4a6f-8463-4417bc19d69f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:24.628843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec470-7ad2-4a2d-ae03-b3d431d6d633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:24.631154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec470-7ad2-4a2d-ae03-b3d431d6d633 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:24.637588 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 126/240] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:24.643402 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec470-7ad2-4a2d-ae03-b3d431d6d633 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:24.690549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:24.692353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.692634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.692904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.693581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:24.693878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:24.695266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:24.713127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 115/241] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:24.720361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:24.821369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:24.824217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec470-7ad2-4a2d-ae03-b3d431d6d633 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:24.839556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:24.887460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76fd53-ad6a-435b-8008-fd51396910df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:24.887980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76fd53-ad6a-435b-8008-fd51396910df None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.888252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76fd53-ad6a-435b-8008-fd51396910df None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.888516 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76fd53-ad6a-435b-8008-fd51396910df None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.889159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76fd53-ad6a-435b-8008-fd51396910df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:24.890620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e76fd53-ad6a-435b-8008-fd51396910df None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:24.893148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d47749-83b4-4a43-b6a7-73c8d746a910 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:24.939304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a2cea-b14f-4c15-a136-9b30dd99f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:24.939946 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a2cea-b14f-4c15-a136-9b30dd99f410 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:24.940221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a2cea-b14f-4c15-a136-9b30dd99f410 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:24.940492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a2cea-b14f-4c15-a136-9b30dd99f410 None admin] PATH_INFO: `/v3/users/330695f9c1fa45afbd4e6653663a2859` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:24.956131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a2cea-b14f-4c15-a136-9b30dd99f410 None admin] RBAC: Authorizing `identity:get_user(user_id=330695f9c1fa45afbd4e6653663a2859)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:24.957583 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a2cea-b14f-4c15-a136-9b30dd99f410 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:24.959490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 116/242] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:24 2022] GET /identity/v3/users/330695f9c1fa45afbd4e6653663a2859 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:24.965339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a2cea-b14f-4c15-a136-9b30dd99f410 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:24.981512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 127/243] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:24 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:03:25.114019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a2cea-b14f-4c15-a136-9b30dd99f410 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:25.149859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:25.177298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47223cb-6ae4-4aa3-ad93-eb5c842a94d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:25.177915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47223cb-6ae4-4aa3-ad93-eb5c842a94d5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.178251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47223cb-6ae4-4aa3-ad93-eb5c842a94d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.178526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47223cb-6ae4-4aa3-ad93-eb5c842a94d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.274209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47223cb-6ae4-4aa3-ad93-eb5c842a94d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:25.275720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47223cb-6ae4-4aa3-ad93-eb5c842a94d5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:25.311622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eba8a5-18d5-4824-b4dd-7533b22eb50c None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.312148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eba8a5-18d5-4824-b4dd-7533b22eb50c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.312148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eba8a5-18d5-4824-b4dd-7533b22eb50c None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.313741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 128/244] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:25.320182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd16db-0d60-4bf8-9be6-be57327299d7 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.320461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd16db-0d60-4bf8-9be6-be57327299d7 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.320740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd16db-0d60-4bf8-9be6-be57327299d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.355890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-59dd16db-0d60-4bf8-9be6-be57327299d7 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:25.357098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-59dd16db-0d60-4bf8-9be6-be57327299d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:25.402308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 117/245] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:25.405623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47223cb-6ae4-4aa3-ad93-eb5c842a94d5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:25.424064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:25.458856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 129/246] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:25.462386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f20f3-3577-4175-9bdd-2ff147baac97 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.462669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f20f3-3577-4175-9bdd-2ff147baac97 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.462940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f20f3-3577-4175-9bdd-2ff147baac97 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.464446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 130/247] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:25.472826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cac134-3165-4762-b64f-fb8ade13822e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.473111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cac134-3165-4762-b64f-fb8ade13822e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.473384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cac134-3165-4762-b64f-fb8ade13822e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.499912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-46cac134-3165-4762-b64f-fb8ade13822e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:25.501440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-46cac134-3165-4762-b64f-fb8ade13822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:25.538332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47223cb-6ae4-4aa3-ad93-eb5c842a94d5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:25.584276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fd0ac-9d81-4673-99b3-41d317ce543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:25.584906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd0ac-9d81-4673-99b3-41d317ce543f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.585184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd0ac-9d81-4673-99b3-41d317ce543f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.585454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd0ac-9d81-4673-99b3-41d317ce543f None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.587796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fd0ac-9d81-4673-99b3-41d317ce543f None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:25.589905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 131/248] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:25.590170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fd0ac-9d81-4673-99b3-41d317ce543f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:25.590633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cac134-3165-4762-b64f-fb8ade13822e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:25.592037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 118/249] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:25 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:25.593302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8450d45-825d-4eba-ace7-368ec37d5828 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.593590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8450d45-825d-4eba-ace7-368ec37d5828 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.593861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8450d45-825d-4eba-ace7-368ec37d5828 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.595402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 119/250] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:25.600732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce47441-a53b-4bad-b9ce-cbcf7e43b488 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.601066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce47441-a53b-4bad-b9ce-cbcf7e43b488 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.601341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce47441-a53b-4bad-b9ce-cbcf7e43b488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.622575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:25.623945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0ce47441-a53b-4bad-b9ce-cbcf7e43b488 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:25.625158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0ce47441-a53b-4bad-b9ce-cbcf7e43b488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:25.702290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 120/251] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:25.703447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce47441-a53b-4bad-b9ce-cbcf7e43b488 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:25.710622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cac134-3165-4762-b64f-fb8ade13822e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:25.719388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:25.767496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77bb66-2b34-456b-8b2a-59ba46779d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:25.768355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77bb66-2b34-456b-8b2a-59ba46779d22 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.768479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77bb66-2b34-456b-8b2a-59ba46779d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.768699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77bb66-2b34-456b-8b2a-59ba46779d22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.781941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77bb66-2b34-456b-8b2a-59ba46779d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-ea77bb66-2b34-456b-8b2a-59ba46779d22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:25.787073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:25.787985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:25.789172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:25.793831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 132/252] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:25.794728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77bb66-2b34-456b-8b2a-59ba46779d22 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:25.816469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce47441-a53b-4bad-b9ce-cbcf7e43b488 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:25.856503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:25.857119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.857402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.857677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.858418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] RBAC: Adding query filter params (role.id=a5998ecc28d44e0c9deb0653af1c8eaf, scope.project.id=456db541138a40ea9d931140527e9c4a, user.id=330695f9c1fa45afbd4e6653663a2859) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:25.858728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:25.860208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:25.866539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 121/253] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:25 2022] GET /identity/v3/role_assignments?user.id=330695f9c1fa45afbd4e6653663a2859&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=a5998ecc28d44e0c9deb0653af1c8eaf => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:25.867356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:25.889000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:25.906782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77bb66-2b34-456b-8b2a-59ba46779d22 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:25.943886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea2b7d-f9f7-48f9-8968-d967c35927bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:25.944522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2b7d-f9f7-48f9-8968-d967c35927bc None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:25.944804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2b7d-f9f7-48f9-8968-d967c35927bc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:25.945078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2b7d-f9f7-48f9-8968-d967c35927bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:25.960131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea2b7d-f9f7-48f9-8968-d967c35927bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6eea2b7d-f9f7-48f9-8968-d967c35927bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:25.962642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:25.964237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:25.965162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 133/254] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:25.966013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea2b7d-f9f7-48f9-8968-d967c35927bc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:25.973111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:26.022308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d59691-46a6-4485-98c3-b2f9274bf524 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:26.076266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea2b7d-f9f7-48f9-8968-d967c35927bc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:26.098037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4d53c-06a0-4dd4-a7d7-b7eb91bbc9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:26.098966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d53c-06a0-4dd4-a7d7-b7eb91bbc9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:26.099417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d53c-06a0-4dd4-a7d7-b7eb91bbc9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:26.099841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d53c-06a0-4dd4-a7d7-b7eb91bbc9d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:26.117946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:26.117946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:26.117946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:26.118290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:26.118739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:26.119067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:26.120554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:26.133617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4d53c-06a0-4dd4-a7d7-b7eb91bbc9d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:26.142017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 134/255] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:26.142898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d5f4d53c-06a0-4dd4-a7d7-b7eb91bbc9d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:26.144888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:26.145773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:26.146611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 122/256] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:26.152457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4d53c-06a0-4dd4-a7d7-b7eb91bbc9d1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:26.158102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:26.259084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8a8-6242-43ce-8f01-7a9e4b6da8d8 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:26.294729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:26.295370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:26.295655 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:26.295938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:26.296606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:26.296913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:26.298315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:26.306903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 135/257] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:26.308471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:26.349174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4d53c-06a0-4dd4-a7d7-b7eb91bbc9d1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:26.375225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:26.410079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbc857-1614-4d97-b303-7718e5f60419 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:26.446002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:26.446617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:26.446898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:26.447202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:26.448395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:26.448702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:26.450095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:26.455119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9767-8c93-413a-8727-42a7bd2f0dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:26.455757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9767-8c93-413a-8727-42a7bd2f0dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:26.456037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9767-8c93-413a-8727-42a7bd2f0dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:26.456311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9767-8c93-413a-8727-42a7bd2f0dc1 None admin] PATH_INFO: `/v3/users/1877997db3804c1c892eee99cebba58a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:26.457447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e9767-8c93-413a-8727-42a7bd2f0dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=1877997db3804c1c892eee99cebba58a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:26.458865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e9767-8c93-413a-8727-42a7bd2f0dc1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:26.468328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 123/258] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:26 2022] GET /identity/v3/users/1877997db3804c1c892eee99cebba58a => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:26.469568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9767-8c93-413a-8727-42a7bd2f0dc1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:26.474005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 136/259] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:26.474005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:26.493304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:26.620162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9767-8c93-413a-8727-42a7bd2f0dc1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:26.663032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd532-9ca1-4ba3-a3d6-f6f244d3356d None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:26.687815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062e14-d142-442b-9521-a17001a8fc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:26.688473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062e14-d142-442b-9521-a17001a8fc06 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:26.688752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062e14-d142-442b-9521-a17001a8fc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:26.689020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062e14-d142-442b-9521-a17001a8fc06 None admin] PATH_INFO: `/v3/users/2acaaf265ad04006b03be9d44d9ad025` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:26.695914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2062e14-d142-442b-9521-a17001a8fc06 None admin] RBAC: Authorizing `identity:get_user(user_id=2acaaf265ad04006b03be9d44d9ad025)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:26.697387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2062e14-d142-442b-9521-a17001a8fc06 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:26.697953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:26.699353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 124/260] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:26 2022] GET /identity/v3/users/2acaaf265ad04006b03be9d44d9ad025 => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:26.699951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062e14-d142-442b-9521-a17001a8fc06 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:26.704096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a655f1-52db-4155-b579-b4ec95ea57d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:26.704758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a655f1-52db-4155-b579-b4ec95ea57d4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:26.705052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a655f1-52db-4155-b579-b4ec95ea57d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:26.705335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a655f1-52db-4155-b579-b4ec95ea57d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:26.839668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a655f1-52db-4155-b579-b4ec95ea57d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:26.842006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a655f1-52db-4155-b579-b4ec95ea57d4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:26.848661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:26.951210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062e14-d142-442b-9521-a17001a8fc06 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:26.957629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 137/261] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:26.958785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a655f1-52db-4155-b579-b4ec95ea57d4 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:27.007428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:27.053180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc295b23-93d6-4747-b5e5-59c16017b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:27.053828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc295b23-93d6-4747-b5e5-59c16017b2fd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.054146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc295b23-93d6-4747-b5e5-59c16017b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.054424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc295b23-93d6-4747-b5e5-59c16017b2fd None admin] PATH_INFO: `/v3/users/1877997db3804c1c892eee99cebba58a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.055590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc295b23-93d6-4747-b5e5-59c16017b2fd None admin] RBAC: Authorizing `identity:get_user(user_id=1877997db3804c1c892eee99cebba58a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:27.057200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc295b23-93d6-4747-b5e5-59c16017b2fd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:27.058662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 125/262] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:26 2022] GET /identity/v3/users/1877997db3804c1c892eee99cebba58a => generated 283 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:27.068084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc295b23-93d6-4747-b5e5-59c16017b2fd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:27.071228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a655f1-52db-4155-b579-b4ec95ea57d4 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:27.123179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14e998-0f08-4463-a920-031a421a653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:27.124163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14e998-0f08-4463-a920-031a421a653f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.124649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14e998-0f08-4463-a920-031a421a653f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.125125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14e998-0f08-4463-a920-031a421a653f None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.126517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14e998-0f08-4463-a920-031a421a653f None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:27.128579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14e998-0f08-4463-a920-031a421a653f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:27.130524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 138/263] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:26 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:27.131553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14e998-0f08-4463-a920-031a421a653f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:27.257243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc295b23-93d6-4747-b5e5-59c16017b2fd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:27.302524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d7b6-4865-42e9-b542-21da7d3f027e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:27.304384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d7b6-4865-42e9-b542-21da7d3f027e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.308701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d7b6-4865-42e9-b542-21da7d3f027e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.309197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d7b6-4865-42e9-b542-21da7d3f027e None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.310605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4d7b6-4865-42e9-b542-21da7d3f027e None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:27.312437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4d7b6-4865-42e9-b542-21da7d3f027e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:27.316017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 126/264] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:27 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:27.318295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d7b6-4865-42e9-b542-21da7d3f027e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:27.332933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:27.337253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14e998-0f08-4463-a920-031a421a653f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:27.351667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:27.409112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480fff0-0382-4882-909e-724e9085a05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:27.410421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fff0-0382-4882-909e-724e9085a05a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.410910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fff0-0382-4882-909e-724e9085a05a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.411413 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fff0-0382-4882-909e-724e9085a05a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.455720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d7b6-4865-42e9-b542-21da7d3f027e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:27.492030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022fa18-3d54-4fef-8ab5-16f33be21ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:27.492704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022fa18-3d54-4fef-8ab5-16f33be21ab3 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.492985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022fa18-3d54-4fef-8ab5-16f33be21ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.493260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022fa18-3d54-4fef-8ab5-16f33be21ab3 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a/users/2acaaf265ad04006b03be9d44d9ad025/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.495814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022fa18-3d54-4fef-8ab5-16f33be21ab3 None admin] RBAC: Authorizing `identity:create_grant(project_id=456db541138a40ea9d931140527e9c4a, user_id=2acaaf265ad04006b03be9d44d9ad025, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:27.497266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022fa18-3d54-4fef-8ab5-16f33be21ab3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:27.532906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480fff0-0382-4882-909e-724e9085a05a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:27.534539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480fff0-0382-4882-909e-724e9085a05a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:27.617981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 127/265] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:27 2022] PUT /identity/v3/projects/456db541138a40ea9d931140527e9c4a/users/2acaaf265ad04006b03be9d44d9ad025/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:27.618859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022fa18-3d54-4fef-8ab5-16f33be21ab3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:27.660889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:27.672191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 139/266] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:27.673814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf5611-242d-43db-8ca8-214d9b533104 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.674105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf5611-242d-43db-8ca8-214d9b533104 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.674377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf5611-242d-43db-8ca8-214d9b533104 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.675662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 140/267] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:27.677989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba40f8-b7de-4b61-a607-054c9f72a1ec None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.678269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba40f8-b7de-4b61-a607-054c9f72a1ec None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.678596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba40f8-b7de-4b61-a607-054c9f72a1ec None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.681823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 141/268] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:27.684398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba40f8-b7de-4b61-a607-054c9f72a1ec None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:27.694473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:27.737893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022fa18-3d54-4fef-8ab5-16f33be21ab3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:27.793799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2a9cd-8b04-4d96-836e-5ee69de77ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:27.794592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a9cd-8b04-4d96-836e-5ee69de77ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.794877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a9cd-8b04-4d96-836e-5ee69de77ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.795181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a9cd-8b04-4d96-836e-5ee69de77ad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.796478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2a9cd-8b04-4d96-836e-5ee69de77ad2 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=456db541138a40ea9d931140527e9c4a, user.id=1877997db3804c1c892eee99cebba58a) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:27.796796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2a9cd-8b04-4d96-836e-5ee69de77ad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:27.798933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2a9cd-8b04-4d96-836e-5ee69de77ad2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:27.806907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba40f8-b7de-4b61-a607-054c9f72a1ec None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:27.807881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 128/269] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:27 2022] GET /identity/v3/role_assignments?user.id=1877997db3804c1c892eee99cebba58a&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=f72d3388f13149b483355331380f040b => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:27.809595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d083a4-e3fd-4c3c-b948-dd0a58086680 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.809926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d083a4-e3fd-4c3c-b948-dd0a58086680 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.810202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d083a4-e3fd-4c3c-b948-dd0a58086680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.836840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b5d083a4-e3fd-4c3c-b948-dd0a58086680 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:27.839186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b5d083a4-e3fd-4c3c-b948-dd0a58086680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:27.867419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e298b20-8d20-43de-96ed-9bcf4220085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:27.868058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e298b20-8d20-43de-96ed-9bcf4220085f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.868331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e298b20-8d20-43de-96ed-9bcf4220085f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.868599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e298b20-8d20-43de-96ed-9bcf4220085f None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.874351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e298b20-8d20-43de-96ed-9bcf4220085f None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:27.877181 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e298b20-8d20-43de-96ed-9bcf4220085f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:27.878659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 142/270] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:27 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:27.882133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06dd0e-ef48-40ca-aaa3-aa79d992ff85 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:27.882133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06dd0e-ef48-40ca-aaa3-aa79d992ff85 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:27.882133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06dd0e-ef48-40ca-aaa3-aa79d992ff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:27.912595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-ff06dd0e-ef48-40ca-aaa3-aa79d992ff85 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:27.914358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ff06dd0e-ef48-40ca-aaa3-aa79d992ff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:27.942904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 129/271] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:27.944149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d083a4-e3fd-4c3c-b948-dd0a58086680 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:28.035593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:28.041373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:28.056569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 143/272] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:28.056569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06dd0e-ef48-40ca-aaa3-aa79d992ff85 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:28.115029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d083a4-e3fd-4c3c-b948-dd0a58086680 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:28.156055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:28.156665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.156943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.157218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.158068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] RBAC: Adding query filter params (role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=456db541138a40ea9d931140527e9c4a, user.id=1877997db3804c1c892eee99cebba58a) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:28.158493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:28.159973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:28.161166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06dd0e-ef48-40ca-aaa3-aa79d992ff85 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:28.168323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 130/273] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:27 2022] GET /identity/v3/role_assignments?user.id=1877997db3804c1c892eee99cebba58a&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:28.169481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:28.235636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff846c3-03c2-405a-89cc-912e6c23e49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:28.236300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff846c3-03c2-405a-89cc-912e6c23e49a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.236579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff846c3-03c2-405a-89cc-912e6c23e49a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.236852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff846c3-03c2-405a-89cc-912e6c23e49a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.263797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff846c3-03c2-405a-89cc-912e6c23e49a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-aff846c3-03c2-405a-89cc-912e6c23e49a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:28.268718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:28.269654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:28.270496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 144/274] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:28.277124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff846c3-03c2-405a-89cc-912e6c23e49a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:28.312328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69256ce-b379-4715-abcb-57572cebdb4d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:28.351136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:28.363074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a83cf3-fe05-45d2-b253-3ab842d4c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:28.365250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a83cf3-fe05-45d2-b253-3ab842d4c91c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.365730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a83cf3-fe05-45d2-b253-3ab842d4c91c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.366180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a83cf3-fe05-45d2-b253-3ab842d4c91c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.370938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a83cf3-fe05-45d2-b253-3ab842d4c91c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:28.370938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a83cf3-fe05-45d2-b253-3ab842d4c91c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:28.372008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 131/275] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:28.381009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a83cf3-fe05-45d2-b253-3ab842d4c91c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:28.407295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:28.457443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff846c3-03c2-405a-89cc-912e6c23e49a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:28.498260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:28.498771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.499092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.499369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.500059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:28.500363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:28.502019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:28.514296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 145/276] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:28.522960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:28.527143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a83cf3-fe05-45d2-b253-3ab842d4c91c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:28.600916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bcc7f-8cda-40f5-a6b6-3d1345ce1fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:28.602034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bcc7f-8cda-40f5-a6b6-3d1345ce1fe8 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.602546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bcc7f-8cda-40f5-a6b6-3d1345ce1fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.603038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bcc7f-8cda-40f5-a6b6-3d1345ce1fe8 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.603953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bcc7f-8cda-40f5-a6b6-3d1345ce1fe8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:28.605750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bcc7f-8cda-40f5-a6b6-3d1345ce1fe8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:28.678489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b7eae-223a-4ac1-8cb5-fc658a949c48 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:28.695855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:28.713144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4431de5-0179-45a7-95a2-9b26b0df6d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:28.714429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4431de5-0179-45a7-95a2-9b26b0df6d0f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.714889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4431de5-0179-45a7-95a2-9b26b0df6d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.715203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4431de5-0179-45a7-95a2-9b26b0df6d0f None admin] PATH_INFO: `/v3/users/b85e072797e54bbca6b238bf695505f7` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.724659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 132/277] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:28 2022] POST /identity/v3/users => generated 293 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:03:28.726536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0d0de-8614-4364-bd85-5cafe897b6c7 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.727045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0d0de-8614-4364-bd85-5cafe897b6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.727525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0d0de-8614-4364-bd85-5cafe897b6c7 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.729340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 133/278] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:28.735312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4431de5-0179-45a7-95a2-9b26b0df6d0f None admin] RBAC: Authorizing `identity:get_user(user_id=b85e072797e54bbca6b238bf695505f7)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:28.735312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4431de5-0179-45a7-95a2-9b26b0df6d0f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:28.735312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 146/279] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:28 2022] GET /identity/v3/users/b85e072797e54bbca6b238bf695505f7 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:28.742662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad20ea8-26eb-469e-8b62-4eaf02b5c3c0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.743268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad20ea8-26eb-469e-8b62-4eaf02b5c3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.747154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad20ea8-26eb-469e-8b62-4eaf02b5c3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.749258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4431de5-0179-45a7-95a2-9b26b0df6d0f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:28.770954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3ad20ea8-26eb-469e-8b62-4eaf02b5c3c0 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:28.773149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3ad20ea8-26eb-469e-8b62-4eaf02b5c3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:28.854739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:28.873248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4431de5-0179-45a7-95a2-9b26b0df6d0f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:28.894616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 134/280] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:28.920366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad20ea8-26eb-469e-8b62-4eaf02b5c3c0 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:28.933675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:28.933675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:28.933675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:28.934993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:28.935268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] RBAC: Adding query filter params (role.id=347174cd91d94848aec704f00988e871, user.id=b85e072797e54bbca6b238bf695505f7, scope.system=all) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:28.936089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:28.937905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:28.945767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 147/281] 10.222.0.30 () {56 vars in 1308 bytes} [Sun Aug 7 12:03:28 2022] GET /identity/v3/role_assignments?user.id=b85e072797e54bbca6b238bf695505f7&scope.system=all&role.id=347174cd91d94848aec704f00988e871 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:29.060947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad20ea8-26eb-469e-8b62-4eaf02b5c3c0 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:29.109385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a7c81-ac0b-466f-b59d-227fbd2923cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:29.109995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a7c81-ac0b-466f-b59d-227fbd2923cc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:29.110277 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a7c81-ac0b-466f-b59d-227fbd2923cc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:29.110550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a7c81-ac0b-466f-b59d-227fbd2923cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:29.123804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a7c81-ac0b-466f-b59d-227fbd2923cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-fc0a7c81-ac0b-466f-b59d-227fbd2923cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:29.128720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:29.129680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:29.130491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 135/282] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:29.137261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:29.190808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:29.258629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e31a4f-dd6e-413e-a848-0532dd88d13e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:29.307323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:29.308512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:29.308996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:29.309522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:29.310396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:29.310923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:29.312557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:29.321802 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 148/283] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:29.331100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a7c81-ac0b-466f-b59d-227fbd2923cc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:29.411336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:29.486260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a7c81-ac0b-466f-b59d-227fbd2923cc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:29.526527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123456-71cd-4c45-9ac7-67f44602499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:29.527292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123456-71cd-4c45-9ac7-67f44602499f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:29.527577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123456-71cd-4c45-9ac7-67f44602499f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:29.527852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123456-71cd-4c45-9ac7-67f44602499f None admin] PATH_INFO: `/v3/users/330695f9c1fa45afbd4e6653663a2859` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:29.528987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25123456-71cd-4c45-9ac7-67f44602499f None admin] RBAC: Authorizing `identity:get_user(user_id=330695f9c1fa45afbd4e6653663a2859)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:29.530420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25123456-71cd-4c45-9ac7-67f44602499f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:29.532211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 136/284] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:29 2022] GET /identity/v3/users/330695f9c1fa45afbd4e6653663a2859 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:29.537438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:29.709000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323519e-a754-4342-98cd-84657d8de4b5 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:29.710485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:29.766120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8bed6-9d3e-476c-88f1-88ec0cb08846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:29.766881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8bed6-9d3e-476c-88f1-88ec0cb08846 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:29.767192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8bed6-9d3e-476c-88f1-88ec0cb08846 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:29.767469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8bed6-9d3e-476c-88f1-88ec0cb08846 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:29.771704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8bed6-9d3e-476c-88f1-88ec0cb08846 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:29.773182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8bed6-9d3e-476c-88f1-88ec0cb08846 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:29.775221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 149/285] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:29 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:29.781201 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123456-71cd-4c45-9ac7-67f44602499f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:29.926682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123456-71cd-4c45-9ac7-67f44602499f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:29.942824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:29.972614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9435d8-57ce-4647-8ae4-80afccb2af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:29.973202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9435d8-57ce-4647-8ae4-80afccb2af21 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:29.973485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9435d8-57ce-4647-8ae4-80afccb2af21 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:29.973763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9435d8-57ce-4647-8ae4-80afccb2af21 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a/users/330695f9c1fa45afbd4e6653663a2859/roles/a5998ecc28d44e0c9deb0653af1c8eaf` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:29.980174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9435d8-57ce-4647-8ae4-80afccb2af21 None admin] RBAC: Authorizing `identity:create_grant(project_id=456db541138a40ea9d931140527e9c4a, user_id=330695f9c1fa45afbd4e6653663a2859, role_id=a5998ecc28d44e0c9deb0653af1c8eaf)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:29.981622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9435d8-57ce-4647-8ae4-80afccb2af21 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:30.054286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 137/286] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:03:29 2022] PUT /identity/v3/projects/456db541138a40ea9d931140527e9c4a/users/330695f9c1fa45afbd4e6653663a2859/roles/a5998ecc28d44e0c9deb0653af1c8eaf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:30.306500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c7a75-ecce-4947-bf64-29da08bc3da9 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:30.306746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c7a75-ecce-4947-bf64-29da08bc3da9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:30.307681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c7a75-ecce-4947-bf64-29da08bc3da9 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:30.309183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 150/287] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:30.315307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270519d-1acf-4b6e-98ca-f517bc1c8a9a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:30.315605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270519d-1acf-4b6e-98ca-f517bc1c8a9a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:30.315887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270519d-1acf-4b6e-98ca-f517bc1c8a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:30.337805 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7270519d-1acf-4b6e-98ca-f517bc1c8a9a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:30.339755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7270519d-1acf-4b6e-98ca-f517bc1c8a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:30.429280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 138/288] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:30.470828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c7a75-ecce-4947-bf64-29da08bc3da9 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:30.607013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c7a75-ecce-4947-bf64-29da08bc3da9 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:30.657276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e47bf1-7bb1-4907-b834-c8855cb10785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:30.657923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e47bf1-7bb1-4907-b834-c8855cb10785 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:30.658205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e47bf1-7bb1-4907-b834-c8855cb10785 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:30.658480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e47bf1-7bb1-4907-b834-c8855cb10785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:30.673651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e47bf1-7bb1-4907-b834-c8855cb10785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-a7e47bf1-7bb1-4907-b834-c8855cb10785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:30.679072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:30.679956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:30.680766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 151/289] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:30.688276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270519d-1acf-4b6e-98ca-f517bc1c8a9a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:30.791871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:30.795389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270519d-1acf-4b6e-98ca-f517bc1c8a9a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:30.832296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:30.832902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:30.833182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:30.833506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:30.834177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:30.834574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:30.835974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:30.849138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 139/290] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:30.851303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e47bf1-7bb1-4907-b834-c8855cb10785 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:30.893568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:30.961890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e47bf1-7bb1-4907-b834-c8855cb10785 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:31.003062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03520e49-4c80-4b2b-a00e-8795158dc055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:31.003746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03520e49-4c80-4b2b-a00e-8795158dc055 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.004037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03520e49-4c80-4b2b-a00e-8795158dc055 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.004308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03520e49-4c80-4b2b-a00e-8795158dc055 None admin] PATH_INFO: `/v3/users/2acaaf265ad04006b03be9d44d9ad025` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.005442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03520e49-4c80-4b2b-a00e-8795158dc055 None admin] RBAC: Authorizing `identity:get_user(user_id=2acaaf265ad04006b03be9d44d9ad025)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:31.006910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03520e49-4c80-4b2b-a00e-8795158dc055 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:31.013083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 152/291] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:30 2022] GET /identity/v3/users/2acaaf265ad04006b03be9d44d9ad025 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:31.014438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:31.127025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6f002-9ac6-4318-aa6e-09e289b95ac9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:31.159921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8895749-5564-422d-bd0f-169d051d66d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:31.160452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8895749-5564-422d-bd0f-169d051d66d6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.160731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8895749-5564-422d-bd0f-169d051d66d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.161006 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8895749-5564-422d-bd0f-169d051d66d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.193209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:31.235117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8895749-5564-422d-bd0f-169d051d66d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:31.236606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8895749-5564-422d-bd0f-169d051d66d6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:31.342204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 140/292] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:31.348053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03520e49-4c80-4b2b-a00e-8795158dc055 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:31.375346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256dc82-fc22-45c1-84e5-6012f21d2d4a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.375672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256dc82-fc22-45c1-84e5-6012f21d2d4a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.375909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256dc82-fc22-45c1-84e5-6012f21d2d4a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.377608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 141/293] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:31.384394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3db1-5c78-4bb4-9ad2-a0b896d8138f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.384802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3db1-5c78-4bb4-9ad2-a0b896d8138f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.385793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3db1-5c78-4bb4-9ad2-a0b896d8138f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.404837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8eaa3db1-5c78-4bb4-9ad2-a0b896d8138f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:31.406166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8eaa3db1-5c78-4bb4-9ad2-a0b896d8138f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:31.468146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:31.479817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 142/294] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:31.500151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2783f-6d54-4e6e-9f58-86a90469b8e9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.500151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2783f-6d54-4e6e-9f58-86a90469b8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.500151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2783f-6d54-4e6e-9f58-86a90469b8e9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.500151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 143/295] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:31.500151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473ca9-6d94-4b12-8b3a-0fd0bc26724e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.500151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473ca9-6d94-4b12-8b3a-0fd0bc26724e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.500570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473ca9-6d94-4b12-8b3a-0fd0bc26724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.506373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:31.515443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-67473ca9-6d94-4b12-8b3a-0fd0bc26724e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:31.516644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-67473ca9-6d94-4b12-8b3a-0fd0bc26724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:31.518485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03520e49-4c80-4b2b-a00e-8795158dc055 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:31.562405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56608b-c14b-49ef-880e-024736b14cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:31.563060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56608b-c14b-49ef-880e-024736b14cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.563339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56608b-c14b-49ef-880e-024736b14cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.563606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56608b-c14b-49ef-880e-024736b14cb2 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.564721 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56608b-c14b-49ef-880e-024736b14cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:31.566155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56608b-c14b-49ef-880e-024736b14cb2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:31.571053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 153/296] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:31 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:31.571053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56608b-c14b-49ef-880e-024736b14cb2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:31.611863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 144/297] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:31.611863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473ca9-6d94-4b12-8b3a-0fd0bc26724e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:31.712999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56608b-c14b-49ef-880e-024736b14cb2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:31.715883 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473ca9-6d94-4b12-8b3a-0fd0bc26724e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:31.742630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:31.756232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73688214-9423-41ff-baba-1dc950370107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:31.756980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73688214-9423-41ff-baba-1dc950370107 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.757394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73688214-9423-41ff-baba-1dc950370107 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.757822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73688214-9423-41ff-baba-1dc950370107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.758736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73688214-9423-41ff-baba-1dc950370107 None admin] RBAC: Adding query filter params (role.id=347174cd91d94848aec704f00988e871, scope.project.id=456db541138a40ea9d931140527e9c4a, user.id=2acaaf265ad04006b03be9d44d9ad025) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:31.759195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73688214-9423-41ff-baba-1dc950370107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:31.761142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148f2f6-c351-465a-9a1b-6dc987c20a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:31.761294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73688214-9423-41ff-baba-1dc950370107 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:31.762041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148f2f6-c351-465a-9a1b-6dc987c20a28 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.762462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148f2f6-c351-465a-9a1b-6dc987c20a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.762872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148f2f6-c351-465a-9a1b-6dc987c20a28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.767900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 145/298] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:31 2022] GET /identity/v3/role_assignments?user.id=2acaaf265ad04006b03be9d44d9ad025&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=347174cd91d94848aec704f00988e871 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:31.771923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73688214-9423-41ff-baba-1dc950370107 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:31.777154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148f2f6-c351-465a-9a1b-6dc987c20a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-2148f2f6-c351-465a-9a1b-6dc987c20a28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:31.782647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:31.783764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:31.784570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:31.787690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 154/299] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:31.796136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148f2f6-c351-465a-9a1b-6dc987c20a28 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:31.849556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:31.908369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73688214-9423-41ff-baba-1dc950370107 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:31.937552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148f2f6-c351-465a-9a1b-6dc987c20a28 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:31.950173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43b98-5d6e-43d1-82b8-c950e764e9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:31.951069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43b98-5d6e-43d1-82b8-c950e764e9cf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.951458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43b98-5d6e-43d1-82b8-c950e764e9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.951866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43b98-5d6e-43d1-82b8-c950e764e9cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.967118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d43b98-5d6e-43d1-82b8-c950e764e9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-58d43b98-5d6e-43d1-82b8-c950e764e9cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:31.973284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:31.974287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:31.975136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 146/300] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:31.976659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ad863-3325-471c-aab5-e399baf92176 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.977084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ad863-3325-471c-aab5-e399baf92176 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.977510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ad863-3325-471c-aab5-e399baf92176 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.983073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 147/301] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:31.983073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ad863-3325-471c-aab5-e399baf92176 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:31.987095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14b590-ce8a-4b74-baf7-a74651cd8ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:31.988257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b590-ce8a-4b74-baf7-a74651cd8ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:31.988744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b590-ce8a-4b74-baf7-a74651cd8ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:31.989244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b590-ce8a-4b74-baf7-a74651cd8ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:31.990366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14b590-ce8a-4b74-baf7-a74651cd8ea4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:31.990952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14b590-ce8a-4b74-baf7-a74651cd8ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:31.992916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14b590-ce8a-4b74-baf7-a74651cd8ea4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:32.004650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 155/302] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:32.006214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4e61a-24ed-46b7-bc6b-7f21fc722bbc None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.006735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4e61a-24ed-46b7-bc6b-7f21fc722bbc None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.008112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4e61a-24ed-46b7-bc6b-7f21fc722bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.028152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-15c4e61a-24ed-46b7-bc6b-7f21fc722bbc None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:32.029590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-15c4e61a-24ed-46b7-bc6b-7f21fc722bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:32.041109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:32.103303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ad863-3325-471c-aab5-e399baf92176 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:32.118256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 156/303] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:32.118256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4e61a-24ed-46b7-bc6b-7f21fc722bbc None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:32.150763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:32.153766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:32.154646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.154646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.154922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.155697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:32.155918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:32.157270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:32.167431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 148/304] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:32.168679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:32.235821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4e61a-24ed-46b7-bc6b-7f21fc722bbc None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:32.270273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51623053-9ce9-4cfc-a29e-821b0a557f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:32.270828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623053-9ce9-4cfc-a29e-821b0a557f1c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.271136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623053-9ce9-4cfc-a29e-821b0a557f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.271410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623053-9ce9-4cfc-a29e-821b0a557f1c None admin] PATH_INFO: `/v3/users/2eaf28520cca41ce8775ffef90a3fb5d` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.281768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5180987-94cd-4b73-ae25-a73ba1120e02 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:32.285107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51623053-9ce9-4cfc-a29e-821b0a557f1c None admin] RBAC: Authorizing `identity:get_user(user_id=2eaf28520cca41ce8775ffef90a3fb5d)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:32.286560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51623053-9ce9-4cfc-a29e-821b0a557f1c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:32.288198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 157/305] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:32 2022] GET /identity/v3/users/2eaf28520cca41ce8775ffef90a3fb5d => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:32.288965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51623053-9ce9-4cfc-a29e-821b0a557f1c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:32.296814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:32.323176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b071e-4a15-448e-a999-bed3dc94ba0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:32.323970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b071e-4a15-448e-a999-bed3dc94ba0e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.324323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b071e-4a15-448e-a999-bed3dc94ba0e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.324598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b071e-4a15-448e-a999-bed3dc94ba0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.325908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b071e-4a15-448e-a999-bed3dc94ba0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:32.327360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b071e-4a15-448e-a999-bed3dc94ba0e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:32.329090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 149/306] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:32.329656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b071e-4a15-448e-a999-bed3dc94ba0e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:32.375299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:32.399024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51623053-9ce9-4cfc-a29e-821b0a557f1c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:32.436359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a58bf7-091b-4e8c-8161-df1ff644820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:32.436997 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58bf7-091b-4e8c-8161-df1ff644820a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.437280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58bf7-091b-4e8c-8161-df1ff644820a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.437554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58bf7-091b-4e8c-8161-df1ff644820a None admin] PATH_INFO: `/v3/users/b85e072797e54bbca6b238bf695505f7` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.438687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a58bf7-091b-4e8c-8161-df1ff644820a None admin] RBAC: Authorizing `identity:get_user(user_id=b85e072797e54bbca6b238bf695505f7)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:32.440164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a58bf7-091b-4e8c-8161-df1ff644820a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:32.446412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 158/307] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:32 2022] GET /identity/v3/users/b85e072797e54bbca6b238bf695505f7 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:32.446412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a58bf7-091b-4e8c-8161-df1ff644820a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:32.446618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b071e-4a15-448e-a999-bed3dc94ba0e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:32.484549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:32.496116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.496116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.496116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.496116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] RBAC: Adding query filter params (role.id=a5998ecc28d44e0c9deb0653af1c8eaf, user.id=2eaf28520cca41ce8775ffef90a3fb5d, scope.system=all) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:32.496116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:32.496116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:32.496595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 150/308] 10.222.0.30 () {56 vars in 1308 bytes} [Sun Aug 7 12:03:32 2022] GET /identity/v3/role_assignments?user.id=2eaf28520cca41ce8775ffef90a3fb5d&scope.system=all&role.id=a5998ecc28d44e0c9deb0653af1c8eaf => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:32.496595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:32.547922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:32.559653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a58bf7-091b-4e8c-8161-df1ff644820a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:32.601064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4814a-907f-442b-8b5e-365c3bbb81cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:32.601687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4814a-907f-442b-8b5e-365c3bbb81cf None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.601965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4814a-907f-442b-8b5e-365c3bbb81cf None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.602238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4814a-907f-442b-8b5e-365c3bbb81cf None admin] PATH_INFO: `/v3/groups` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.602866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4814a-907f-442b-8b5e-365c3bbb81cf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:32.605007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4814a-907f-442b-8b5e-365c3bbb81cf None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:32.617896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb20cfb-fcc7-48fd-afce-c0d11466692a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:32.654513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb9da8-9c6d-43b8-851b-5a8e6179d22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:32.655202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9da8-9c6d-43b8-851b-5a8e6179d22e None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.655487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9da8-9c6d-43b8-851b-5a8e6179d22e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.662441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9da8-9c6d-43b8-851b-5a8e6179d22e None admin] PATH_INFO: `/v3/system/users/b85e072797e54bbca6b238bf695505f7/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.662441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb9da8-9c6d-43b8-851b-5a8e6179d22e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b85e072797e54bbca6b238bf695505f7, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:32.662441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb9da8-9c6d-43b8-851b-5a8e6179d22e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:32.670792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 159/309] 10.222.0.30 () {60 vars in 1134 bytes} [Sun Aug 7 12:03:32 2022] POST /identity/v3/groups => generated 229 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:32.672059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f68f9-f372-461a-bce4-5c1b7a722f7a None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.672419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f68f9-f372-461a-bce4-5c1b7a722f7a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.672640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f68f9-f372-461a-bce4-5c1b7a722f7a None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.674046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 160/310] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:32.679529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e086-0542-4fb4-9f28-3c51aa14677a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.679808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e086-0542-4fb4-9f28-3c51aa14677a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.680090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e086-0542-4fb4-9f28-3c51aa14677a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.694553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:32.701971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-b1d7e086-0542-4fb4-9f28-3c51aa14677a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:32.702991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b1d7e086-0542-4fb4-9f28-3c51aa14677a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:32.725019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 151/311] 10.222.0.30 () {58 vars in 1334 bytes} [Sun Aug 7 12:03:32 2022] PUT /identity/v3/system/users/b85e072797e54bbca6b238bf695505f7/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:32.794088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 161/312] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:32.816590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb9da8-9c6d-43b8-851b-5a8e6179d22e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:32.933274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb9da8-9c6d-43b8-851b-5a8e6179d22e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:32.936693 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:32.975715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e603fcd-99c3-45c1-a0f6-4d6e0c695664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:32.976819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603fcd-99c3-45c1-a0f6-4d6e0c695664 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:32.977352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603fcd-99c3-45c1-a0f6-4d6e0c695664 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:32.977812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603fcd-99c3-45c1-a0f6-4d6e0c695664 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:32.989931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e603fcd-99c3-45c1-a0f6-4d6e0c695664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3e603fcd-99c3-45c1-a0f6-4d6e0c695664 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:32.995834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:32.996923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:32.997729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:33.000323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 152/313] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:33.005722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e086-0542-4fb4-9f28-3c51aa14677a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:33.112475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e086-0542-4fb4-9f28-3c51aa14677a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:33.150835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:33.152049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:33.152550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:33.153016 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:33.153884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:33.154379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:33.155990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:33.157608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:33.166818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 162/314] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:33.176063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e603fcd-99c3-45c1-a0f6-4d6e0c695664 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:33.284724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e603fcd-99c3-45c1-a0f6-4d6e0c695664 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:33.320353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6f396-6fb5-4439-a703-8510cba0f117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:33.321312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6f396-6fb5-4439-a703-8510cba0f117 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:33.321817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6f396-6fb5-4439-a703-8510cba0f117 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:33.322284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6f396-6fb5-4439-a703-8510cba0f117 None admin] PATH_INFO: `/v3/users/330695f9c1fa45afbd4e6653663a2859` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:33.324223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6f396-6fb5-4439-a703-8510cba0f117 None admin] RBAC: Authorizing `identity:get_user(user_id=330695f9c1fa45afbd4e6653663a2859)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:33.325863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6f396-6fb5-4439-a703-8510cba0f117 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:33.327968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 153/315] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:33 2022] GET /identity/v3/users/330695f9c1fa45afbd4e6653663a2859 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:33.333658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:33.434572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d1fb3-8fb2-4d8d-8b52-9e2203d7098a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:33.470045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb1b37-0a93-4059-8c05-e89dc5ceab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:33.470660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1b37-0a93-4059-8c05-e89dc5ceab82 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:33.470946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1b37-0a93-4059-8c05-e89dc5ceab82 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:33.471246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1b37-0a93-4059-8c05-e89dc5ceab82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:33.536093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:33.542890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb1b37-0a93-4059-8c05-e89dc5ceab82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:33.544434 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb1b37-0a93-4059-8c05-e89dc5ceab82 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:33.648202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 163/316] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:33.653825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6f396-6fb5-4439-a703-8510cba0f117 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:33.695791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:33.769199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6f396-6fb5-4439-a703-8510cba0f117 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:33.804225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eec1d7-03d9-44e8-86fa-3d13097303a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:33.804857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec1d7-03d9-44e8-86fa-3d13097303a7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:33.805143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec1d7-03d9-44e8-86fa-3d13097303a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:33.805421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec1d7-03d9-44e8-86fa-3d13097303a7 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:33.806555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eec1d7-03d9-44e8-86fa-3d13097303a7 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:33.808327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eec1d7-03d9-44e8-86fa-3d13097303a7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:33.810417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 154/317] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:33 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:33.816092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb1b37-0a93-4059-8c05-e89dc5ceab82 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:33.917478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:33.928281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb1b37-0a93-4059-8c05-e89dc5ceab82 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:33.969688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f388d7-5c00-4d43-b6b4-d30728128625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:33.970218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f388d7-5c00-4d43-b6b4-d30728128625 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:33.970502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f388d7-5c00-4d43-b6b4-d30728128625 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:33.970781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f388d7-5c00-4d43-b6b4-d30728128625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:33.971554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f388d7-5c00-4d43-b6b4-d30728128625 None admin] RBAC: Adding query filter params (role.id=a5998ecc28d44e0c9deb0653af1c8eaf, scope.project.id=456db541138a40ea9d931140527e9c4a, user.id=330695f9c1fa45afbd4e6653663a2859) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:33.971873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f388d7-5c00-4d43-b6b4-d30728128625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:33.973319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f388d7-5c00-4d43-b6b4-d30728128625 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:33.980703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 164/318] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:03:33 2022] GET /identity/v3/role_assignments?user.id=330695f9c1fa45afbd4e6653663a2859&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=a5998ecc28d44e0c9deb0653af1c8eaf => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:34.588845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be0942-5490-4e93-99d8-986dff18dab9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:34.589226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be0942-5490-4e93-99d8-986dff18dab9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:34.589402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be0942-5490-4e93-99d8-986dff18dab9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:34.590856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 155/319] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:34.595893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaa453-4e3c-41fb-91b5-484933c8794a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:34.596180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaa453-4e3c-41fb-91b5-484933c8794a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:34.596455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaa453-4e3c-41fb-91b5-484933c8794a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:34.614933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-82eaa453-4e3c-41fb-91b5-484933c8794a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:34.616142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-82eaa453-4e3c-41fb-91b5-484933c8794a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:34.680838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 165/320] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:34.702724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be0942-5490-4e93-99d8-986dff18dab9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:34.800642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:34.805011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be0942-5490-4e93-99d8-986dff18dab9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:34.828116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f691d-f601-4feb-a33f-74abf5d57551 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:34.828609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f691d-f601-4feb-a33f-74abf5d57551 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:34.828719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f691d-f601-4feb-a33f-74abf5d57551 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:34.830097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 166/321] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:34.834889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3be781-bebd-4706-8a3f-b54afbf2c772 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:34.835209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3be781-bebd-4706-8a3f-b54afbf2c772 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:34.835489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3be781-bebd-4706-8a3f-b54afbf2c772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:34.840839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19454a-c66e-4fca-a9a9-bdaa5a7efd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:34.841387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19454a-c66e-4fca-a9a9-bdaa5a7efd6f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:34.841664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19454a-c66e-4fca-a9a9-bdaa5a7efd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:34.841930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19454a-c66e-4fca-a9a9-bdaa5a7efd6f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:34.852803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19454a-c66e-4fca-a9a9-bdaa5a7efd6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:34.854702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-aa3be781-bebd-4706-8a3f-b54afbf2c772 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:34.856054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-aa3be781-bebd-4706-8a3f-b54afbf2c772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-da19454a-c66e-4fca-a9a9-bdaa5a7efd6f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:34.858162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:34.859097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:34.859938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 156/322] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:34.867112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19454a-c66e-4fca-a9a9-bdaa5a7efd6f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:34.927574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 167/323] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:34.949998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3be781-bebd-4706-8a3f-b54afbf2c772 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:34.972301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:34.973936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19454a-c66e-4fca-a9a9-bdaa5a7efd6f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:35.011067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d40e4e-2091-497b-829e-6b265e1bca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:35.011671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d40e4e-2091-497b-829e-6b265e1bca8d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.011949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d40e4e-2091-497b-829e-6b265e1bca8d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.012222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d40e4e-2091-497b-829e-6b265e1bca8d None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.012884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d40e4e-2091-497b-829e-6b265e1bca8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:35.013183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d40e4e-2091-497b-829e-6b265e1bca8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:35.014588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d40e4e-2091-497b-829e-6b265e1bca8d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:35.022555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 157/324] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:35.024249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00215b15-dca4-4f4d-bf97-f8df37afcba6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.024249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00215b15-dca4-4f4d-bf97-f8df37afcba6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.024249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00215b15-dca4-4f4d-bf97-f8df37afcba6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.025418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 158/325] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:35.030382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00215b15-dca4-4f4d-bf97-f8df37afcba6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:35.054091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3be781-bebd-4706-8a3f-b54afbf2c772 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:35.054548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:35.087344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158108a8-a4d9-489a-8c3d-d06d710e4a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:35.087975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158108a8-a4d9-489a-8c3d-d06d710e4a5e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.088876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158108a8-a4d9-489a-8c3d-d06d710e4a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.089371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158108a8-a4d9-489a-8c3d-d06d710e4a5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.090582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158108a8-a4d9-489a-8c3d-d06d710e4a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:35.092134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158108a8-a4d9-489a-8c3d-d06d710e4a5e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:35.093639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 168/326] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:35.094810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8e33e-524a-4578-9496-399d25e1ee45 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.095122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8e33e-524a-4578-9496-399d25e1ee45 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.095400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8e33e-524a-4578-9496-399d25e1ee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.114038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-26f8e33e-524a-4578-9496-399d25e1ee45 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:35.115297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-26f8e33e-524a-4578-9496-399d25e1ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:35.134527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00215b15-dca4-4f4d-bf97-f8df37afcba6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:35.169551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a7cf9-e6c9-4316-87d4-b8955c84812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:35.170183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a7cf9-e6c9-4316-87d4-b8955c84812b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.170461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a7cf9-e6c9-4316-87d4-b8955c84812b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.170736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a7cf9-e6c9-4316-87d4-b8955c84812b None admin] PATH_INFO: `/v3/users/2eaf28520cca41ce8775ffef90a3fb5d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.171887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a7cf9-e6c9-4316-87d4-b8955c84812b None admin] RBAC: Authorizing `identity:get_user(user_id=2eaf28520cca41ce8775ffef90a3fb5d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:35.173316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a7cf9-e6c9-4316-87d4-b8955c84812b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:35.174880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 159/327] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:35 2022] GET /identity/v3/users/2eaf28520cca41ce8775ffef90a3fb5d => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:35.179811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a7cf9-e6c9-4316-87d4-b8955c84812b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:35.182780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 169/328] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:35.183621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8e33e-524a-4578-9496-399d25e1ee45 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:35.201183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:35.277538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:35.281926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8e33e-524a-4578-9496-399d25e1ee45 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:35.286613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a7cf9-e6c9-4316-87d4-b8955c84812b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:35.316976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa979b-baae-4a5b-874d-cd68a672bb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:35.317622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa979b-baae-4a5b-874d-cd68a672bb88 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.317898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa979b-baae-4a5b-874d-cd68a672bb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.318183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa979b-baae-4a5b-874d-cd68a672bb88 None admin] PATH_INFO: `/v3/system/users/2eaf28520cca41ce8775ffef90a3fb5d/roles/a5998ecc28d44e0c9deb0653af1c8eaf` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.319591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa979b-baae-4a5b-874d-cd68a672bb88 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2eaf28520cca41ce8775ffef90a3fb5d, role_id=a5998ecc28d44e0c9deb0653af1c8eaf)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:35.320959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa979b-baae-4a5b-874d-cd68a672bb88 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:35.323464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4a824-b68a-4fc2-8e65-5885c9e087bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:35.324056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a824-b68a-4fc2-8e65-5885c9e087bb None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.324336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a824-b68a-4fc2-8e65-5885c9e087bb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.324731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a824-b68a-4fc2-8e65-5885c9e087bb None admin] PATH_INFO: `/v3/groups` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.325345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4a824-b68a-4fc2-8e65-5885c9e087bb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:35.326758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4a824-b68a-4fc2-8e65-5885c9e087bb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:35.384904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 170/329] 10.222.0.30 () {58 vars in 1334 bytes} [Sun Aug 7 12:03:35 2022] PUT /identity/v3/system/users/2eaf28520cca41ce8775ffef90a3fb5d/roles/a5998ecc28d44e0c9deb0653af1c8eaf => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:35.386014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa979b-baae-4a5b-874d-cd68a672bb88 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:35.417779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 160/330] 10.222.0.30 () {60 vars in 1134 bytes} [Sun Aug 7 12:03:35 2022] POST /identity/v3/groups => generated 226 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:35.481685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:35.502899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa979b-baae-4a5b-874d-cd68a672bb88 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:35.545642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8ed4e-7ec7-4db4-bc2c-624a670c7f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:35.546871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8ed4e-7ec7-4db4-bc2c-624a670c7f24 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.547496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8ed4e-7ec7-4db4-bc2c-624a670c7f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.548148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8ed4e-7ec7-4db4-bc2c-624a670c7f24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.566590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8ed4e-7ec7-4db4-bc2c-624a670c7f24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-d2e8ed4e-7ec7-4db4-bc2c-624a670c7f24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:35.572073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:35.573987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:35.574795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 171/331] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:35.580339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4a824-b68a-4fc2-8e65-5885c9e087bb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:35.692582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4a824-b68a-4fc2-8e65-5885c9e087bb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:35.729570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:35.730179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.730434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.730706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.731413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:35.731713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:35.733082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:35.742619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 161/332] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:35.749006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8ed4e-7ec7-4db4-bc2c-624a670c7f24 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:35.853250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8ed4e-7ec7-4db4-bc2c-624a670c7f24 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:35.894475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154cd5f-4aa4-4d30-9fb3-984e8b307574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:35.895156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154cd5f-4aa4-4d30-9fb3-984e8b307574 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:35.895437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154cd5f-4aa4-4d30-9fb3-984e8b307574 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:35.895714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154cd5f-4aa4-4d30-9fb3-984e8b307574 None admin] PATH_INFO: `/v3/users/b85e072797e54bbca6b238bf695505f7` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:35.896879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154cd5f-4aa4-4d30-9fb3-984e8b307574 None admin] RBAC: Authorizing `identity:get_user(user_id=b85e072797e54bbca6b238bf695505f7)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:35.898361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154cd5f-4aa4-4d30-9fb3-984e8b307574 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:35.899994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 172/333] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:35 2022] GET /identity/v3/users/b85e072797e54bbca6b238bf695505f7 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:35.905974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:35.924257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:36.019780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc6713-4422-4e0d-8a1e-a391a2cd9a1e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:36.061136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767932ed-0987-4478-9302-a903d2b3ed4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:36.061745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767932ed-0987-4478-9302-a903d2b3ed4f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:36.062026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767932ed-0987-4478-9302-a903d2b3ed4f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:36.062298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767932ed-0987-4478-9302-a903d2b3ed4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:36.063098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767932ed-0987-4478-9302-a903d2b3ed4f None admin] RBAC: Adding query filter params (role.id=347174cd91d94848aec704f00988e871, user.id=b85e072797e54bbca6b238bf695505f7, scope.system=all) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:36.063409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767932ed-0987-4478-9302-a903d2b3ed4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:36.064833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767932ed-0987-4478-9302-a903d2b3ed4f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:36.081003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 162/334] 10.222.0.30 () {56 vars in 1308 bytes} [Sun Aug 7 12:03:35 2022] GET /identity/v3/role_assignments?user.id=b85e072797e54bbca6b238bf695505f7&scope.system=all&role.id=347174cd91d94848aec704f00988e871 => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:37.501244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944307-6356-4cc9-ac81-589ea82a1ed1 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:37.501814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944307-6356-4cc9-ac81-589ea82a1ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:37.501814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944307-6356-4cc9-ac81-589ea82a1ed1 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:37.503400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 173/335] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:37.508759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b08ec9-c852-4348-878d-9c3b45e7efed None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:37.509200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b08ec9-c852-4348-878d-9c3b45e7efed None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:37.509290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b08ec9-c852-4348-878d-9c3b45e7efed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:37.531635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-79b08ec9-c852-4348-878d-9c3b45e7efed None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:37.532865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-79b08ec9-c852-4348-878d-9c3b45e7efed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:37.620166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:37.625223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 163/336] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:37.664333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7944307-6356-4cc9-ac81-589ea82a1ed1 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:37.678169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:37.774914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7944307-6356-4cc9-ac81-589ea82a1ed1 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:37.811325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c6e0c-50ce-4ea5-a7fd-a7efe5ea5cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:37.811955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c6e0c-50ce-4ea5-a7fd-a7efe5ea5cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:37.812231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c6e0c-50ce-4ea5-a7fd-a7efe5ea5cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:37.812501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c6e0c-50ce-4ea5-a7fd-a7efe5ea5cc7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:37.823330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c6e0c-50ce-4ea5-a7fd-a7efe5ea5cc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-f22c6e0c-50ce-4ea5-a7fd-a7efe5ea5cc7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:37.828570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:37.829461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 12:03:37.830281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:37.834541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 174/337] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:37.835927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b08ec9-c852-4348-878d-9c3b45e7efed None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:37.954117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fc47-3be3-412e-8016-5446e5944e0b None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:37.954435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fc47-3be3-412e-8016-5446e5944e0b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:37.954659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fc47-3be3-412e-8016-5446e5944e0b None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:37.956007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 175/338] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:37.967840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b08ec9-c852-4348-878d-9c3b45e7efed None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:37.968539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18ef0-d55c-4379-8ece-64d90b55f90e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:37.968825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18ef0-d55c-4379-8ece-64d90b55f90e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:37.969100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c18ef0-d55c-4379-8ece-64d90b55f90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:37.994816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f3c18ef0-d55c-4379-8ece-64d90b55f90e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:37.996485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f3c18ef0-d55c-4379-8ece-64d90b55f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:38.050570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8c200-5482-41df-91c0-48d4662e7ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:38.051128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8c200-5482-41df-91c0-48d4662e7ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.051515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8c200-5482-41df-91c0-48d4662e7ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.051797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8c200-5482-41df-91c0-48d4662e7ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.052472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8c200-5482-41df-91c0-48d4662e7ff8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:38.052778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8c200-5482-41df-91c0-48d4662e7ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:38.054293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8c200-5482-41df-91c0-48d4662e7ff8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:38.065239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 164/339] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:38.066777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b8a0a-9887-4a89-94f7-0ffb62e300d0 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.067351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b8a0a-9887-4a89-94f7-0ffb62e300d0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.067977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b8a0a-9887-4a89-94f7-0ffb62e300d0 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.069399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 165/340] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:38.075923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b8a0a-9887-4a89-94f7-0ffb62e300d0 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:38.130073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:38.135781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 176/341] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:38.137033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4bb3-f6df-4413-906b-5b87c24be4cd None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.137317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4bb3-f6df-4413-906b-5b87c24be4cd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.137590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4bb3-f6df-4413-906b-5b87c24be4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.167648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-9e0c4bb3-f6df-4413-906b-5b87c24be4cd None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:38.169138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9e0c4bb3-f6df-4413-906b-5b87c24be4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:38.212722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:38.273676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 177/342] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:38.274645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4bb3-f6df-4413-906b-5b87c24be4cd None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:38.287395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b8a0a-9887-4a89-94f7-0ffb62e300d0 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:38.335805 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea393652-f0ac-4766-be22-ae51c666b85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:38.336798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea393652-f0ac-4766-be22-ae51c666b85f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.337357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea393652-f0ac-4766-be22-ae51c666b85f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.337834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea393652-f0ac-4766-be22-ae51c666b85f None admin] PATH_INFO: `/v3/users/2eaf28520cca41ce8775ffef90a3fb5d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.339734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea393652-f0ac-4766-be22-ae51c666b85f None admin] RBAC: Authorizing `identity:get_user(user_id=2eaf28520cca41ce8775ffef90a3fb5d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:38.341529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea393652-f0ac-4766-be22-ae51c666b85f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:38.343482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 166/343] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:03:38 2022] GET /identity/v3/users/2eaf28520cca41ce8775ffef90a3fb5d => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:38.344462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea393652-f0ac-4766-be22-ae51c666b85f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:38.433320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4bb3-f6df-4413-906b-5b87c24be4cd None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:38.462920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea393652-f0ac-4766-be22-ae51c666b85f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:38.501691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d881fa3-dd8e-48bc-8323-7f6b6aa65a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:38.502337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881fa3-dd8e-48bc-8323-7f6b6aa65a70 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.502619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881fa3-dd8e-48bc-8323-7f6b6aa65a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.502894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881fa3-dd8e-48bc-8323-7f6b6aa65a70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.514490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:38.515871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d881fa3-dd8e-48bc-8323-7f6b6aa65a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4d881fa3-dd8e-48bc-8323-7f6b6aa65a70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:38.521443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:38.530045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:38.530874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:38.531551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:38.531941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 167/344] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:38.531941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d881fa3-dd8e-48bc-8323-7f6b6aa65a70 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:38.546925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb100f-4b0d-4928-928f-acde61a3c597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:38.547624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb100f-4b0d-4928-928f-acde61a3c597 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.547901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb100f-4b0d-4928-928f-acde61a3c597 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.548168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb100f-4b0d-4928-928f-acde61a3c597 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.585916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb100f-4b0d-4928-928f-acde61a3c597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6adb100f-4b0d-4928-928f-acde61a3c597 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:38.591451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:38.592391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:38.593309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 178/345] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:38.595784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb100f-4b0d-4928-928f-acde61a3c597 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:38.639748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d881fa3-dd8e-48bc-8323-7f6b6aa65a70 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:38.677060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:38.677675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.677961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.678292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.679049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] RBAC: Adding query filter params (role.id=a5998ecc28d44e0c9deb0653af1c8eaf, user.id=2eaf28520cca41ce8775ffef90a3fb5d, scope.system=all) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:38.679420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:38.680864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:38.694097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 168/346] 10.222.0.30 () {56 vars in 1308 bytes} [Sun Aug 7 12:03:38 2022] GET /identity/v3/role_assignments?user.id=2eaf28520cca41ce8775ffef90a3fb5d&scope.system=all&role.id=a5998ecc28d44e0c9deb0653af1c8eaf => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:38.694928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:38.750468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:38.807421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f540f-c327-490c-8c47-b4920388e682 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:38.847393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:38.847994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.848605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.848890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.849581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:38.849886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:38.851300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:38.862246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 169/347] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:38.869045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:38.886374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb100f-4b0d-4928-928f-acde61a3c597 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:38.978137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:38.979738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:38.980056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:38.980327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:38.981003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:38.981303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:38.982711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:38.992643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd049128-00bc-477b-87a9-c7ce4c0fbc67 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:38.997397 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 179/348] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:39.002212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:39.004421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:39.037163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add63fb7-c7d4-4610-8507-e80495525b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:39.037785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63fb7-c7d4-4610-8507-e80495525b05 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.038065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63fb7-c7d4-4610-8507-e80495525b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.038338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63fb7-c7d4-4610-8507-e80495525b05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.079053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:39.169128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63fb7-c7d4-4610-8507-e80495525b05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:39.170581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63fb7-c7d4-4610-8507-e80495525b05 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:39.187115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149297b4-7db8-4a6a-aee0-e9a5a699dfcd None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:39.239127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0dd17-087c-4513-bbd3-e4c8ae47f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:39.239999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0dd17-087c-4513-bbd3-e4c8ae47f7ef None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.240681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0dd17-087c-4513-bbd3-e4c8ae47f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.241097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0dd17-087c-4513-bbd3-e4c8ae47f7ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.359167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0dd17-087c-4513-bbd3-e4c8ae47f7ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:39.360566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0dd17-087c-4513-bbd3-e4c8ae47f7ef None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:39.376017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 170/349] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:39.377653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7812fdf-d534-45a9-a4a5-0ee4d42eff12 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.382688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7812fdf-d534-45a9-a4a5-0ee4d42eff12 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.383216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7812fdf-d534-45a9-a4a5-0ee4d42eff12 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.384737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 171/350] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:39.386298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7812fdf-d534-45a9-a4a5-0ee4d42eff12 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:39.401067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:39.429135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:39.500670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 180/351] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:39.501856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f995-ff8a-4adb-8f75-2089afa9747f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.502146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f995-ff8a-4adb-8f75-2089afa9747f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.502424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f995-ff8a-4adb-8f75-2089afa9747f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.512535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7812fdf-d534-45a9-a4a5-0ee4d42eff12 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:39.524200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1a29f995-ff8a-4adb-8f75-2089afa9747f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:39.526502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1a29f995-ff8a-4adb-8f75-2089afa9747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:39.568637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677d409-7c0c-48d0-8c6b-07c5695d1b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:39.569858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677d409-7c0c-48d0-8c6b-07c5695d1b6d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.570139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677d409-7c0c-48d0-8c6b-07c5695d1b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.570409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677d409-7c0c-48d0-8c6b-07c5695d1b6d None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.572021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677d409-7c0c-48d0-8c6b-07c5695d1b6d None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:39.574296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677d409-7c0c-48d0-8c6b-07c5695d1b6d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:39.575718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 172/352] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:39 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:39.576489 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677d409-7c0c-48d0-8c6b-07c5695d1b6d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:39.613563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 181/353] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:39.615319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29f995-ff8a-4adb-8f75-2089afa9747f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:39.659982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:39.685854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677d409-7c0c-48d0-8c6b-07c5695d1b6d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:39.708239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:39.723250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61439626-d066-42bd-912e-db3a8a40de88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:39.723896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61439626-d066-42bd-912e-db3a8a40de88 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.724207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61439626-d066-42bd-912e-db3a8a40de88 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.724429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61439626-d066-42bd-912e-db3a8a40de88 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.725574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61439626-d066-42bd-912e-db3a8a40de88 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:39.726983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61439626-d066-42bd-912e-db3a8a40de88 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:39.729495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 173/354] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:39 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:39.730293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61439626-d066-42bd-912e-db3a8a40de88 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:39.731212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29f995-ff8a-4adb-8f75-2089afa9747f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:39.771803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802c66a-d68e-49df-b393-f24fa0b757c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:39.772845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802c66a-d68e-49df-b393-f24fa0b757c6 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.773349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802c66a-d68e-49df-b393-f24fa0b757c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.773846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802c66a-d68e-49df-b393-f24fa0b757c6 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.780877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802c66a-d68e-49df-b393-f24fa0b757c6 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:39.782750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802c66a-d68e-49df-b393-f24fa0b757c6 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:39.784943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 182/355] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:39 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:39.785993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802c66a-d68e-49df-b393-f24fa0b757c6 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:39.845933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61439626-d066-42bd-912e-db3a8a40de88 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:39.892787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff351d-99ee-44f4-a15c-a6cd4d2da8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:39.893845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff351d-99ee-44f4-a15c-a6cd4d2da8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.894135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff351d-99ee-44f4-a15c-a6cd4d2da8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.894574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff351d-99ee-44f4-a15c-a6cd4d2da8d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.913675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff351d-99ee-44f4-a15c-a6cd4d2da8d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:39.913970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802c66a-d68e-49df-b393-f24fa0b757c6 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c6ff351d-99ee-44f4-a15c-a6cd4d2da8d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:39.920399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:39.921359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:39.922173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 174/356] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:39.923750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c82da-5fca-4adb-b942-a89b1bae9e4e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.924131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c82da-5fca-4adb-b942-a89b1bae9e4e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.924328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c82da-5fca-4adb-b942-a89b1bae9e4e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.926626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 175/357] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:39.926626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c82da-5fca-4adb-b942-a89b1bae9e4e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:39.936342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:39.965453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f99d51-9138-4a89-aae8-940a6a9079d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:39.966427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f99d51-9138-4a89-aae8-940a6a9079d4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:39.966974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f99d51-9138-4a89-aae8-940a6a9079d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:39.967448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f99d51-9138-4a89-aae8-940a6a9079d4 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:39.968618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f99d51-9138-4a89-aae8-940a6a9079d4 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:39.970372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f99d51-9138-4a89-aae8-940a6a9079d4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:39.972646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 183/358] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:39 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:39.973338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f99d51-9138-4a89-aae8-940a6a9079d4 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:39.993366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:40.053037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c82da-5fca-4adb-b942-a89b1bae9e4e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:40.106685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f99d51-9138-4a89-aae8-940a6a9079d4 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:40.113491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4acd08-bdb4-4a22-b5d5-3e9c011cd971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:40.114103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4acd08-bdb4-4a22-b5d5-3e9c011cd971 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.114378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4acd08-bdb4-4a22-b5d5-3e9c011cd971 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.114646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4acd08-bdb4-4a22-b5d5-3e9c011cd971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.117506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4acd08-bdb4-4a22-b5d5-3e9c011cd971 None admin] RBAC: Adding query filter params (group.id=71f3b4903a4641a9a945faee7df59e99, role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=5d73bd77f96146698091e497e8e76566) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:40.117506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4acd08-bdb4-4a22-b5d5-3e9c011cd971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:40.117506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4acd08-bdb4-4a22-b5d5-3e9c011cd971 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:40.128601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 176/359] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:39 2022] GET /identity/v3/role_assignments?group.id=71f3b4903a4641a9a945faee7df59e99&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:40.128848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f31f4d-48bb-40ae-a359-47c5aaa1cac2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.129046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f31f4d-48bb-40ae-a359-47c5aaa1cac2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.129227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f31f4d-48bb-40ae-a359-47c5aaa1cac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.154787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-45f31f4d-48bb-40ae-a359-47c5aaa1cac2 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:40.154787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-45f31f4d-48bb-40ae-a359-47c5aaa1cac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:40.174116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:40.174840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.175242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.175585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.176488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] RBAC: Adding query filter params (name=member) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:40.177210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:40.178657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:40.195227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 184/360] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:40.197482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:40.255584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:40.261429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 177/361] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:40.262232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f31f4d-48bb-40ae-a359-47c5aaa1cac2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:40.302922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:40.332295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d181-52e9-48b1-9bbf-8982acaf5028 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:40.376793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63356cf-8a91-4e6c-b556-3c56f9a62b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:40.377835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63356cf-8a91-4e6c-b556-3c56f9a62b75 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.377835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63356cf-8a91-4e6c-b556-3c56f9a62b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.378037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63356cf-8a91-4e6c-b556-3c56f9a62b75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.378724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63356cf-8a91-4e6c-b556-3c56f9a62b75 None admin] RBAC: Adding query filter params (group.id=71f3b4903a4641a9a945faee7df59e99, role.id=347174cd91d94848aec704f00988e871, scope.project.id=5d73bd77f96146698091e497e8e76566) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:40.380732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63356cf-8a91-4e6c-b556-3c56f9a62b75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:40.380732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63356cf-8a91-4e6c-b556-3c56f9a62b75 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:40.388006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 185/362] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:40 2022] GET /identity/v3/role_assignments?group.id=71f3b4903a4641a9a945faee7df59e99&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=347174cd91d94848aec704f00988e871 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:40.389238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c663-7c24-4ee2-9e34-af6117ddb0ef None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.389516 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c663-7c24-4ee2-9e34-af6117ddb0ef None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.390058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031c663-7c24-4ee2-9e34-af6117ddb0ef None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.391051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 186/363] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:40.396083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031c663-7c24-4ee2-9e34-af6117ddb0ef None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:40.396922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f31f4d-48bb-40ae-a359-47c5aaa1cac2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:40.438642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8a1cb-ebe2-4d0b-82c6-2203671a08cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:40.439641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8a1cb-ebe2-4d0b-82c6-2203671a08cc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.440154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8a1cb-ebe2-4d0b-82c6-2203671a08cc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.440631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8a1cb-ebe2-4d0b-82c6-2203671a08cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.519477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:40.521971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8a1cb-ebe2-4d0b-82c6-2203671a08cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:40.524874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8a1cb-ebe2-4d0b-82c6-2203671a08cc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:40.531989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031c663-7c24-4ee2-9e34-af6117ddb0ef None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:40.575650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292efe5-85c8-45fd-be6a-b0740bfc1b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:40.576552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292efe5-85c8-45fd-be6a-b0740bfc1b0d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.576904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292efe5-85c8-45fd-be6a-b0740bfc1b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.577393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292efe5-85c8-45fd-be6a-b0740bfc1b0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.599395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:40.605259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292efe5-85c8-45fd-be6a-b0740bfc1b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-0292efe5-85c8-45fd-be6a-b0740bfc1b0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:40.610622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:40.611816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:40.613115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:40.614389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 187/364] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:40.616689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a40c2d-b71c-4f3f-8c38-c17ee5e380a7 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.617045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a40c2d-b71c-4f3f-8c38-c17ee5e380a7 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.617382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a40c2d-b71c-4f3f-8c38-c17ee5e380a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.641596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-77a40c2d-b71c-4f3f-8c38-c17ee5e380a7 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:40.643047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-77a40c2d-b71c-4f3f-8c38-c17ee5e380a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:40.653751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 178/365] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:40.654694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8a1cb-ebe2-4d0b-82c6-2203671a08cc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:40.736958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 188/366] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:40.738341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a40c2d-b71c-4f3f-8c38-c17ee5e380a7 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:40.795805 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:40.798714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8a1cb-ebe2-4d0b-82c6-2203671a08cc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:40.842424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf731bc-0691-432a-869b-92759285becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:40.845626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf731bc-0691-432a-869b-92759285becf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.845626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf731bc-0691-432a-869b-92759285becf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.845626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf731bc-0691-432a-869b-92759285becf None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.845626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf731bc-0691-432a-869b-92759285becf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:40.845626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf731bc-0691-432a-869b-92759285becf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:40.845996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf731bc-0691-432a-869b-92759285becf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:40.855745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 179/367] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:40.856686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf731bc-0691-432a-869b-92759285becf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:40.861174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a40c2d-b71c-4f3f-8c38-c17ee5e380a7 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:40.898127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:40.906459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4c737-5565-4bb9-be82-23e69e32a214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:40.907731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4c737-5565-4bb9-be82-23e69e32a214 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:40.908076 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4c737-5565-4bb9-be82-23e69e32a214 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:40.908775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4c737-5565-4bb9-be82-23e69e32a214 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:40.910261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4c737-5565-4bb9-be82-23e69e32a214 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:40.912226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4c737-5565-4bb9-be82-23e69e32a214 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:40.914500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 189/368] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:40 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:40.915687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4c737-5565-4bb9-be82-23e69e32a214 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:40.973356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf731bc-0691-432a-869b-92759285becf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.016218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f171b-c707-4fb3-b087-3fbb8406c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.016780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f171b-c707-4fb3-b087-3fbb8406c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.017063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f171b-c707-4fb3-b087-3fbb8406c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.017338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f171b-c707-4fb3-b087-3fbb8406c3a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.030572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f171b-c707-4fb3-b087-3fbb8406c3a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-069f171b-c707-4fb3-b087-3fbb8406c3a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:41.035927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:41.036815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:41.037626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 180/369] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:41.038290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f171b-c707-4fb3-b087-3fbb8406c3a6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:41.042786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4c737-5565-4bb9-be82-23e69e32a214 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.077673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:41.087242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd598b8-d3b9-4fab-bc8d-9158c5052306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.088923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd598b8-d3b9-4fab-bc8d-9158c5052306 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.089421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd598b8-d3b9-4fab-bc8d-9158c5052306 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.089864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd598b8-d3b9-4fab-bc8d-9158c5052306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.184556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f171b-c707-4fb3-b087-3fbb8406c3a6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.189127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:41.193470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd598b8-d3b9-4fab-bc8d-9158c5052306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:41.195657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd598b8-d3b9-4fab-bc8d-9158c5052306 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:41.231343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415b7f-628d-4cd5-a8b0-383dca355396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.231917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415b7f-628d-4cd5-a8b0-383dca355396 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.232165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415b7f-628d-4cd5-a8b0-383dca355396 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.232441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415b7f-628d-4cd5-a8b0-383dca355396 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.233641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85415b7f-628d-4cd5-a8b0-383dca355396 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:41.235371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85415b7f-628d-4cd5-a8b0-383dca355396 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:41.242255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 181/370] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:41 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:41.242255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415b7f-628d-4cd5-a8b0-383dca355396 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:41.346721 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 190/371] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:41.347433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd598b8-d3b9-4fab-bc8d-9158c5052306 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:41.363361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:41.372046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415b7f-628d-4cd5-a8b0-383dca355396 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.408465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.409072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.409350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.409625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.410297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:41.410600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:41.412010 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:41.424298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 182/372] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:41.426600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:41.458238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:41.463079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd598b8-d3b9-4fab-bc8d-9158c5052306 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.511206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.511981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.512269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.512544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.513287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] RBAC: Adding query filter params (group.id=71f3b4903a4641a9a945faee7df59e99, role.id=347174cd91d94848aec704f00988e871, scope.project.id=456db541138a40ea9d931140527e9c4a) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:41.513594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:41.515266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:41.524392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 191/373] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:41 2022] GET /identity/v3/role_assignments?group.id=71f3b4903a4641a9a945faee7df59e99&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=347174cd91d94848aec704f00988e871 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:41.525927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:41.566455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944eec1-08a9-4f29-9883-3969e327b401 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.613146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e628572-21d5-43a1-b46f-0233b66d59aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.613867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e628572-21d5-43a1-b46f-0233b66d59aa None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.614992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e628572-21d5-43a1-b46f-0233b66d59aa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.615293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e628572-21d5-43a1-b46f-0233b66d59aa None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.616443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e628572-21d5-43a1-b46f-0233b66d59aa None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:41.617879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e628572-21d5-43a1-b46f-0233b66d59aa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:41.620307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 183/374] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:41 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:41.625541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e628572-21d5-43a1-b46f-0233b66d59aa None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:41.651390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:41.654507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44bd6d-cfcd-457e-801b-0646c57b880a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.694810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f168e5-6fda-4890-a5f9-d4077adbac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.695471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f168e5-6fda-4890-a5f9-d4077adbac12 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.695758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f168e5-6fda-4890-a5f9-d4077adbac12 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.696035 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f168e5-6fda-4890-a5f9-d4077adbac12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.732903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:41.739119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e628572-21d5-43a1-b46f-0233b66d59aa None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.777914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c0112-ec42-41ac-85ab-c62718a272f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.778556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c0112-ec42-41ac-85ab-c62718a272f5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.778833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c0112-ec42-41ac-85ab-c62718a272f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.779202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c0112-ec42-41ac-85ab-c62718a272f5 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.780317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c0112-ec42-41ac-85ab-c62718a272f5 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:41.781780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c0112-ec42-41ac-85ab-c62718a272f5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:41.782007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f168e5-6fda-4890-a5f9-d4077adbac12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:41.783584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f168e5-6fda-4890-a5f9-d4077adbac12 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:41.783784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 184/375] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:41 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:41.789547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c0112-ec42-41ac-85ab-c62718a272f5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:41.908685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 192/376] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:41.912220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:41.915862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f168e5-6fda-4890-a5f9-d4077adbac12 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:41.926904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c0112-ec42-41ac-85ab-c62718a272f5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:41.971761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:41.972370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:41.972655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:41.972929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:41.973664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] RBAC: Adding query filter params (group.id=71f3b4903a4641a9a945faee7df59e99, role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=456db541138a40ea9d931140527e9c4a) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:41.973975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:41.975435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:41.991048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 185/377] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:41 2022] GET /identity/v3/role_assignments?group.id=71f3b4903a4641a9a945faee7df59e99&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:42.023998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:42.065638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f168e5-6fda-4890-a5f9-d4077adbac12 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:42.111136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60e5b3-799e-49fd-9cb0-6b8f4c399226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:42.115111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60e5b3-799e-49fd-9cb0-6b8f4c399226 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:42.115111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60e5b3-799e-49fd-9cb0-6b8f4c399226 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:42.115111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60e5b3-799e-49fd-9cb0-6b8f4c399226 None admin] PATH_INFO: `/v3/projects/7cc097a340cd405e932b3797d5864d76` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:42.118468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60e5b3-799e-49fd-9cb0-6b8f4c399226 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc097a340cd405e932b3797d5864d76)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:42.118468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60e5b3-799e-49fd-9cb0-6b8f4c399226 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:42.119138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 193/378] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:41 2022] GET /identity/v3/projects/7cc097a340cd405e932b3797d5864d76 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:42.127030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:42.267752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecb032-b177-4581-b9a1-adab494c4c53 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:42.308807 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115583c9-5233-4ac2-9b0c-0622fa00b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:42.309455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115583c9-5233-4ac2-9b0c-0622fa00b3fc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:42.309741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115583c9-5233-4ac2-9b0c-0622fa00b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:42.310017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115583c9-5233-4ac2-9b0c-0622fa00b3fc None admin] PATH_INFO: `/v3/groups/63428a825943497f8c7cc02efa056d05` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:42.317339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115583c9-5233-4ac2-9b0c-0622fa00b3fc None admin] RBAC: Authorizing `identity:get_group(group_id=63428a825943497f8c7cc02efa056d05)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:42.318783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115583c9-5233-4ac2-9b0c-0622fa00b3fc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:42.320806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 186/379] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:42 2022] GET /identity/v3/groups/63428a825943497f8c7cc02efa056d05 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:42.326364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60e5b3-799e-49fd-9cb0-6b8f4c399226 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:42.438206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60e5b3-799e-49fd-9cb0-6b8f4c399226 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:42.471340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:42.482087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f01ed-4b54-4cd8-b102-45522104551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:42.482703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f01ed-4b54-4cd8-b102-45522104551f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:42.491853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f01ed-4b54-4cd8-b102-45522104551f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:42.491853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f01ed-4b54-4cd8-b102-45522104551f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:42.491853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f01ed-4b54-4cd8-b102-45522104551f None admin] RBAC: Adding query filter params (group.id=63428a825943497f8c7cc02efa056d05, role.id=f72d3388f13149b483355331380f040b, scope.project.id=7cc097a340cd405e932b3797d5864d76) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:42.491853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f01ed-4b54-4cd8-b102-45522104551f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:42.491853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f01ed-4b54-4cd8-b102-45522104551f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:42.492218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 194/380] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:42 2022] GET /identity/v3/role_assignments?group.id=63428a825943497f8c7cc02efa056d05&scope.project.id=7cc097a340cd405e932b3797d5864d76&role.id=f72d3388f13149b483355331380f040b => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:42.741636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3044344-cc65-4a8a-a316-35ab7ac2fbd6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:42.742665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3044344-cc65-4a8a-a316-35ab7ac2fbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:42.743206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3044344-cc65-4a8a-a316-35ab7ac2fbd6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:42.745241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 187/381] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:42.750632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32e081-1351-4ef5-a852-b16a3e2d8f54 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:42.751628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32e081-1351-4ef5-a852-b16a3e2d8f54 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:42.752111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32e081-1351-4ef5-a852-b16a3e2d8f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:42.779766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2c32e081-1351-4ef5-a852-b16a3e2d8f54 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:42.781050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2c32e081-1351-4ef5-a852-b16a3e2d8f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:42.869859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 195/382] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:42.912448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3044344-cc65-4a8a-a316-35ab7ac2fbd6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:42.924841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce54a3-c77c-4649-9177-b00e74876fa8 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:42.927620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce54a3-c77c-4649-9177-b00e74876fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:42.928187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce54a3-c77c-4649-9177-b00e74876fa8 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:42.929773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 196/383] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:42.936282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bccbd5-9511-49f2-acac-8ab416fa6e43 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:42.936854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bccbd5-9511-49f2-acac-8ab416fa6e43 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:42.937331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bccbd5-9511-49f2-acac-8ab416fa6e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:42.947119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:42.980151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-53bccbd5-9511-49f2-acac-8ab416fa6e43 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:42.983390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-53bccbd5-9511-49f2-acac-8ab416fa6e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:43.108158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3044344-cc65-4a8a-a316-35ab7ac2fbd6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:43.139269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 197/384] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:43.169111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7ec5e-f29a-47c1-97a1-8ae4fa91fc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:43.169766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7ec5e-f29a-47c1-97a1-8ae4fa91fc85 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:43.170051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7ec5e-f29a-47c1-97a1-8ae4fa91fc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:43.170328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7ec5e-f29a-47c1-97a1-8ae4fa91fc85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:43.179109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bccbd5-9511-49f2-acac-8ab416fa6e43 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:43.203287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7ec5e-f29a-47c1-97a1-8ae4fa91fc85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1cd7ec5e-f29a-47c1-97a1-8ae4fa91fc85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:43.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:43.212084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:43.212882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 188/385] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:43.218980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7ec5e-f29a-47c1-97a1-8ae4fa91fc85 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:43.261867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:43.317357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bccbd5-9511-49f2-acac-8ab416fa6e43 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:43.362492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd63c61-b2d2-41a7-9cbb-f267f3357a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:43.363325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd63c61-b2d2-41a7-9cbb-f267f3357a5c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:43.363610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd63c61-b2d2-41a7-9cbb-f267f3357a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:43.363887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd63c61-b2d2-41a7-9cbb-f267f3357a5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:43.389703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd63c61-b2d2-41a7-9cbb-f267f3357a5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-3dd63c61-b2d2-41a7-9cbb-f267f3357a5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:43.395313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:43.396203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:43.397094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:43.397723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:43.397801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 198/386] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:43.403110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd63c61-b2d2-41a7-9cbb-f267f3357a5c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:43.433473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7ec5e-f29a-47c1-97a1-8ae4fa91fc85 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:43.488580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:43.489204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:43.489491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:43.489768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:43.490452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:43.490756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:43.492160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:43.507061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 189/387] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:43.513144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:43.517299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd63c61-b2d2-41a7-9cbb-f267f3357a5c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:43.533228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:43.558182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:43.559193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:43.559809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:43.560280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:43.561208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:43.561718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:43.563391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:43.581130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 199/388] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:43.592107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:43.708918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0bdf1-192f-4421-a605-6bcb156d53b7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:43.781725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a76a-b39b-4a75-90e3-3755ff7baa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:43.782475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a76a-b39b-4a75-90e3-3755ff7baa11 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:43.782638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a76a-b39b-4a75-90e3-3755ff7baa11 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:43.782914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a76a-b39b-4a75-90e3-3755ff7baa11 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:43.785012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358a76a-b39b-4a75-90e3-3755ff7baa11 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:43.786439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358a76a-b39b-4a75-90e3-3755ff7baa11 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:43.792961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 190/389] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:43 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:43.794929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a76a-b39b-4a75-90e3-3755ff7baa11 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:43.809757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:43.812943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8cdba-4bcb-49b7-8587-19021de3070b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:43.859779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf2909-4841-4976-9b33-f76b977691f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:43.860427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf2909-4841-4976-9b33-f76b977691f2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:43.860760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf2909-4841-4976-9b33-f76b977691f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:43.861040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf2909-4841-4976-9b33-f76b977691f2 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:43.863151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf2909-4841-4976-9b33-f76b977691f2 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:43.864683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf2909-4841-4976-9b33-f76b977691f2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:43.867614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 200/390] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:43 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:43.874783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf2909-4841-4976-9b33-f76b977691f2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:43.974858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:43.996943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a76a-b39b-4a75-90e3-3755ff7baa11 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:44.067835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10a883-7fa7-4ff5-aede-fd0accd48caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:44.068487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a883-7fa7-4ff5-aede-fd0accd48caf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.068771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a883-7fa7-4ff5-aede-fd0accd48caf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.069147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a883-7fa7-4ff5-aede-fd0accd48caf None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.070366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10a883-7fa7-4ff5-aede-fd0accd48caf None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:44.071814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10a883-7fa7-4ff5-aede-fd0accd48caf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:44.073868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 191/391] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:43 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:44.078923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf2909-4841-4976-9b33-f76b977691f2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:44.080274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10a883-7fa7-4ff5-aede-fd0accd48caf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:44.128842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650f5b-728b-4478-9660-bae0feaf0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:44.129534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650f5b-728b-4478-9660-bae0feaf0f95 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.129818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650f5b-728b-4478-9660-bae0feaf0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.130144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650f5b-728b-4478-9660-bae0feaf0f95 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.132191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe650f5b-728b-4478-9660-bae0feaf0f95 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:44.133662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe650f5b-728b-4478-9660-bae0feaf0f95 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:44.136316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 201/392] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:43 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:44.145812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650f5b-728b-4478-9660-bae0feaf0f95 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:44.243170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:44.287831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650f5b-728b-4478-9660-bae0feaf0f95 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:44.304949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:44.321169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10a883-7fa7-4ff5-aede-fd0accd48caf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:44.329231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320bf67-318b-455f-893c-5589c5b90cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:44.330065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320bf67-318b-455f-893c-5589c5b90cca None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.330271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320bf67-318b-455f-893c-5589c5b90cca None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.330524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320bf67-318b-455f-893c-5589c5b90cca None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566/groups/71f3b4903a4641a9a945faee7df59e99/roles/9475450bfdd04912abb88ccf9973f9d8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.339043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320bf67-318b-455f-893c-5589c5b90cca None admin] RBAC: Authorizing `identity:create_grant(project_id=5d73bd77f96146698091e497e8e76566, group_id=71f3b4903a4641a9a945faee7df59e99, role_id=9475450bfdd04912abb88ccf9973f9d8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:44.339043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320bf67-318b-455f-893c-5589c5b90cca None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:44.374226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4129-cf72-47ac-8a1a-68c5a3a31e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:44.374945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4129-cf72-47ac-8a1a-68c5a3a31e81 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.375262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4129-cf72-47ac-8a1a-68c5a3a31e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.375543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4129-cf72-47ac-8a1a-68c5a3a31e81 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566/groups/71f3b4903a4641a9a945faee7df59e99/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.377747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f4129-cf72-47ac-8a1a-68c5a3a31e81 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d73bd77f96146698091e497e8e76566, group_id=71f3b4903a4641a9a945faee7df59e99, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:44.379310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f4129-cf72-47ac-8a1a-68c5a3a31e81 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:44.402979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 202/393] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 12:03:44 2022] PUT /identity/v3/projects/5d73bd77f96146698091e497e8e76566/groups/71f3b4903a4641a9a945faee7df59e99/roles/9475450bfdd04912abb88ccf9973f9d8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:44.404135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb5eb0-d323-466c-afde-f258d7a9c9ab None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.404426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb5eb0-d323-466c-afde-f258d7a9c9ab None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.404705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb5eb0-d323-466c-afde-f258d7a9c9ab None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.406189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 203/394] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:44.412028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3670-39b3-4e4b-a20c-bf350b1faa69 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.412373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3670-39b3-4e4b-a20c-bf350b1faa69 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.412654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3670-39b3-4e4b-a20c-bf350b1faa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.434206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-458d3670-39b3-4e4b-a20c-bf350b1faa69 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:44.435493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-458d3670-39b3-4e4b-a20c-bf350b1faa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:44.465784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 192/395] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 12:03:44 2022] PUT /identity/v3/projects/5d73bd77f96146698091e497e8e76566/groups/71f3b4903a4641a9a945faee7df59e99/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:03:44.543070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 204/396] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:44.576846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4129-cf72-47ac-8a1a-68c5a3a31e81 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:44.610325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0925b92-8f68-4891-8174-86a755e9b518 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.610632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0925b92-8f68-4891-8174-86a755e9b518 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.610884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0925b92-8f68-4891-8174-86a755e9b518 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.612287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 205/397] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:44.620606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412914ee-2949-46e4-adc6-df4b30cfee97 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.620606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412914ee-2949-46e4-adc6-df4b30cfee97 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.620857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412914ee-2949-46e4-adc6-df4b30cfee97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.647475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-412914ee-2949-46e4-adc6-df4b30cfee97 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:44.648688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-412914ee-2949-46e4-adc6-df4b30cfee97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:44.719894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4129-cf72-47ac-8a1a-68c5a3a31e81 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:44.749294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 206/398] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:44.772798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dddf2-8553-411f-922b-5f151fea7869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:44.773432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dddf2-8553-411f-922b-5f151fea7869 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.773713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dddf2-8553-411f-922b-5f151fea7869 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.773986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dddf2-8553-411f-922b-5f151fea7869 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.776566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412914ee-2949-46e4-adc6-df4b30cfee97 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:44.780793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dddf2-8553-411f-922b-5f151fea7869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7d4dddf2-8553-411f-922b-5f151fea7869 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:44.811766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:44.813215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:44.814135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 193/399] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:44.814955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dddf2-8553-411f-922b-5f151fea7869 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:44.823785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:44.927091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412914ee-2949-46e4-adc6-df4b30cfee97 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:44.952764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dddf2-8553-411f-922b-5f151fea7869 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:44.963820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f971fa6-1540-48e1-bc6d-d5137a9a8136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:44.964718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f971fa6-1540-48e1-bc6d-d5137a9a8136 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.964718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f971fa6-1540-48e1-bc6d-d5137a9a8136 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.965025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f971fa6-1540-48e1-bc6d-d5137a9a8136 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:44.975563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f971fa6-1540-48e1-bc6d-d5137a9a8136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-0f971fa6-1540-48e1-bc6d-d5137a9a8136 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:44.981218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:44.982139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:44.983424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 207/400] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:44.988505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f971fa6-1540-48e1-bc6d-d5137a9a8136 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:44.997114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:44.998030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:44.998605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:44.999106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.000171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:45.000770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:45.002510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:45.013208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 194/401] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:45.023819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:45.091672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:45.106929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f971fa6-1540-48e1-bc6d-d5137a9a8136 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:45.109217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:45.148658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:45.149679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.150189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.151350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.152477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:45.152990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:45.154593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:45.162260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49ec52-ed70-40fd-9b64-da4167cc15a3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:45.165538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 208/402] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:45.172703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:45.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652f01d-28e0-4729-950d-26b3795dc869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:45.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652f01d-28e0-4729-950d-26b3795dc869 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652f01d-28e0-4729-950d-26b3795dc869 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.205381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652f01d-28e0-4729-950d-26b3795dc869 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.206267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652f01d-28e0-4729-950d-26b3795dc869 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:45.207743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652f01d-28e0-4729-950d-26b3795dc869 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:45.211510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 195/403] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:45 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:45.218900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652f01d-28e0-4729-950d-26b3795dc869 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:45.320122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67d257-bf66-487e-ba87-a5fda56a268e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:45.320739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652f01d-28e0-4729-950d-26b3795dc869 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:45.345827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:45.356987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06508045-8b50-466e-abb6-ba4510aa8ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:45.357793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508045-8b50-466e-abb6-ba4510aa8ebe None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.358134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508045-8b50-466e-abb6-ba4510aa8ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.358477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508045-8b50-466e-abb6-ba4510aa8ebe None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.359696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06508045-8b50-466e-abb6-ba4510aa8ebe None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:45.361180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06508045-8b50-466e-abb6-ba4510aa8ebe None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:45.363444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 196/404] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:45 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:45.363786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587a29e-3362-439d-8c7b-9ab1bd8d6061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:45.364476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587a29e-3362-439d-8c7b-9ab1bd8d6061 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.364846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587a29e-3362-439d-8c7b-9ab1bd8d6061 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.365125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587a29e-3362-439d-8c7b-9ab1bd8d6061 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.366246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587a29e-3362-439d-8c7b-9ab1bd8d6061 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:45.367728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587a29e-3362-439d-8c7b-9ab1bd8d6061 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:45.369922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa8643-5260-4ceb-a1a1-65b56c1f479a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.370277 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa8643-5260-4ceb-a1a1-65b56c1f479a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.370619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa8643-5260-4ceb-a1a1-65b56c1f479a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.370812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 209/405] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:45 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:45.371549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587a29e-3362-439d-8c7b-9ab1bd8d6061 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:45.372427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 197/406] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:45.378225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abdb6d-5fb6-4cb9-84f7-c34e04c755ea None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.378712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abdb6d-5fb6-4cb9-84f7-c34e04c755ea None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.379075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abdb6d-5fb6-4cb9-84f7-c34e04c755ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.397660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f1abdb6d-5fb6-4cb9-84f7-c34e04c755ea None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:45.398939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f1abdb6d-5fb6-4cb9-84f7-c34e04c755ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:45.402322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:45.468299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 198/407] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:45.469163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abdb6d-5fb6-4cb9-84f7-c34e04c755ea None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:45.504381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587a29e-3362-439d-8c7b-9ab1bd8d6061 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:45.547676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883e7d9-4e28-4c1c-bff2-42231976809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:45.548398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883e7d9-4e28-4c1c-bff2-42231976809b None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.548680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883e7d9-4e28-4c1c-bff2-42231976809b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.548952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883e7d9-4e28-4c1c-bff2-42231976809b None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a/groups/71f3b4903a4641a9a945faee7df59e99/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.550985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883e7d9-4e28-4c1c-bff2-42231976809b None admin] RBAC: Authorizing `identity:create_grant(project_id=456db541138a40ea9d931140527e9c4a, group_id=71f3b4903a4641a9a945faee7df59e99, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:45.552580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883e7d9-4e28-4c1c-bff2-42231976809b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:45.583544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abdb6d-5fb6-4cb9-84f7-c34e04c755ea None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:45.609086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:45.621520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547d802-8745-44fb-a9d1-d0a577061166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:45.622146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547d802-8745-44fb-a9d1-d0a577061166 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.622422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547d802-8745-44fb-a9d1-d0a577061166 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.622691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547d802-8745-44fb-a9d1-d0a577061166 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.623829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547d802-8745-44fb-a9d1-d0a577061166 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:45.625393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547d802-8745-44fb-a9d1-d0a577061166 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:45.627202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 199/408] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:45 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:45.627781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547d802-8745-44fb-a9d1-d0a577061166 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:45.699386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 210/409] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 12:03:45 2022] PUT /identity/v3/projects/456db541138a40ea9d931140527e9c4a/groups/71f3b4903a4641a9a945faee7df59e99/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:45.701206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883e7d9-4e28-4c1c-bff2-42231976809b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:45.760248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547d802-8745-44fb-a9d1-d0a577061166 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:45.842612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13b1ce-7953-42b5-a42b-ad984955888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:45.844323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13b1ce-7953-42b5-a42b-ad984955888a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:45.845645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13b1ce-7953-42b5-a42b-ad984955888a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:45.846728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13b1ce-7953-42b5-a42b-ad984955888a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:45.873325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13b1ce-7953-42b5-a42b-ad984955888a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4b13b1ce-7953-42b5-a42b-ad984955888a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:45.887030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:45.888038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:45.888857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 200/410] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:45.889553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13b1ce-7953-42b5-a42b-ad984955888a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:45.897583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:45.934105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883e7d9-4e28-4c1c-bff2-42231976809b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:45.975684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:45.999889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c5c99-0167-4c6c-88e1-340f54f3a0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:46.000932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c5c99-0167-4c6c-88e1-340f54f3a0df None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:46.001415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c5c99-0167-4c6c-88e1-340f54f3a0df None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:46.001891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c5c99-0167-4c6c-88e1-340f54f3a0df None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a/groups/71f3b4903a4641a9a945faee7df59e99/roles/9475450bfdd04912abb88ccf9973f9d8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:46.008409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c5c99-0167-4c6c-88e1-340f54f3a0df None admin] RBAC: Authorizing `identity:create_grant(project_id=456db541138a40ea9d931140527e9c4a, group_id=71f3b4903a4641a9a945faee7df59e99, role_id=9475450bfdd04912abb88ccf9973f9d8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:46.010304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c5c99-0167-4c6c-88e1-340f54f3a0df None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:46.035644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13b1ce-7953-42b5-a42b-ad984955888a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:46.077097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 211/411] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 12:03:45 2022] PUT /identity/v3/projects/456db541138a40ea9d931140527e9c4a/groups/71f3b4903a4641a9a945faee7df59e99/roles/9475450bfdd04912abb88ccf9973f9d8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:46.080633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:46.081523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:46.081806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:46.082080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:46.082765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:46.087100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:46.087100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:46.098992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 201/412] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:46.100429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c5c99-0167-4c6c-88e1-340f54f3a0df None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:46.225231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c5c99-0167-4c6c-88e1-340f54f3a0df None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:46.262379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a041f44-e1d4-482d-887c-eac2e4889594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:46.263042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a041f44-e1d4-482d-887c-eac2e4889594 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:46.263316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a041f44-e1d4-482d-887c-eac2e4889594 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:46.263590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a041f44-e1d4-482d-887c-eac2e4889594 None admin] PATH_INFO: `/v3/groups/63428a825943497f8c7cc02efa056d05` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:46.264706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a041f44-e1d4-482d-887c-eac2e4889594 None admin] RBAC: Authorizing `identity:get_group(group_id=63428a825943497f8c7cc02efa056d05)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:46.266150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a041f44-e1d4-482d-887c-eac2e4889594 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:46.267935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 212/413] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:46 2022] GET /identity/v3/groups/63428a825943497f8c7cc02efa056d05 => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:46.273219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:46.393068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece512e-238a-48d7-951a-ebb6d72931b7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:46.411165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:46.438940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc22706-25b8-4e06-bc34-014a6bc1e03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:46.439927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc22706-25b8-4e06-bc34-014a6bc1e03e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:46.440443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc22706-25b8-4e06-bc34-014a6bc1e03e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:46.440897 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc22706-25b8-4e06-bc34-014a6bc1e03e None admin] PATH_INFO: `/v3/projects/7cc097a340cd405e932b3797d5864d76` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:46.442203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc22706-25b8-4e06-bc34-014a6bc1e03e None admin] RBAC: Authorizing `identity:get_project(project_id=7cc097a340cd405e932b3797d5864d76)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:46.443845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc22706-25b8-4e06-bc34-014a6bc1e03e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:46.445658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 202/414] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:46 2022] GET /identity/v3/projects/7cc097a340cd405e932b3797d5864d76 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:46.452337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a041f44-e1d4-482d-887c-eac2e4889594 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:46.470569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:46.573542 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a041f44-e1d4-482d-887c-eac2e4889594 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:46.609899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfabfd7-4331-4e69-941e-622cb79fcc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:46.610563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfabfd7-4331-4e69-941e-622cb79fcc78 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:46.610851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfabfd7-4331-4e69-941e-622cb79fcc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:46.611163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfabfd7-4331-4e69-941e-622cb79fcc78 None admin] PATH_INFO: `/v3/projects/7cc097a340cd405e932b3797d5864d76/groups/63428a825943497f8c7cc02efa056d05/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:46.613346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfabfd7-4331-4e69-941e-622cb79fcc78 None admin] RBAC: Authorizing `identity:create_grant(project_id=7cc097a340cd405e932b3797d5864d76, group_id=63428a825943497f8c7cc02efa056d05, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:46.614806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfabfd7-4331-4e69-941e-622cb79fcc78 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:46.684989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 213/415] 10.222.0.30 () {58 vars in 1442 bytes} [Sun Aug 7 12:03:46 2022] PUT /identity/v3/projects/7cc097a340cd405e932b3797d5864d76/groups/63428a825943497f8c7cc02efa056d05/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:03:46.951882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf38b-5d85-4562-8e6d-f6169202fb8c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:46.952393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf38b-5d85-4562-8e6d-f6169202fb8c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:46.952659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf38b-5d85-4562-8e6d-f6169202fb8c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:46.953857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 203/416] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:46.959560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db065b-22b5-45d3-8c15-bee274c088c3 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:46.959991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db065b-22b5-45d3-8c15-bee274c088c3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:46.960166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db065b-22b5-45d3-8c15-bee274c088c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:46.980744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-85db065b-22b5-45d3-8c15-bee274c088c3 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:46.981934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-85db065b-22b5-45d3-8c15-bee274c088c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:47.050703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:47.055431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 214/417] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:47.071285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a866-b5c3-4e8e-9400-b8b726526e2a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:47.071571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a866-b5c3-4e8e-9400-b8b726526e2a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:47.071843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a866-b5c3-4e8e-9400-b8b726526e2a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:47.073401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 204/418] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:47.078688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb658d1-5bee-4026-a1f7-31a4ebe60bfb None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:47.078970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb658d1-5bee-4026-a1f7-31a4ebe60bfb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:47.079311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb658d1-5bee-4026-a1f7-31a4ebe60bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:47.082131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9a866-b5c3-4e8e-9400-b8b726526e2a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:47.107255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2bb658d1-5bee-4026-a1f7-31a4ebe60bfb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:47.108525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2bb658d1-5bee-4026-a1f7-31a4ebe60bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:47.207395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9a866-b5c3-4e8e-9400-b8b726526e2a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:47.248310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 215/419] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:47.253412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313fc90-6a3d-40f9-8a78-ce3a981e7068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:47.254304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313fc90-6a3d-40f9-8a78-ce3a981e7068 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:47.254780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313fc90-6a3d-40f9-8a78-ce3a981e7068 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:47.255297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313fc90-6a3d-40f9-8a78-ce3a981e7068 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:47.267158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313fc90-6a3d-40f9-8a78-ce3a981e7068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d313fc90-6a3d-40f9-8a78-ce3a981e7068 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:47.272447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:47.273328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:47.274162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:47.277345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 205/420] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:47.284195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb658d1-5bee-4026-a1f7-31a4ebe60bfb None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:47.288765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313fc90-6a3d-40f9-8a78-ce3a981e7068 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:47.383425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:47.457930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb658d1-5bee-4026-a1f7-31a4ebe60bfb None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:47.461257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313fc90-6a3d-40f9-8a78-ce3a981e7068 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:47.510261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:47.518610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9c9c9-3fdd-4f8c-b8ce-51a424884f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:47.519634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c9c9-3fdd-4f8c-b8ce-51a424884f6a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:47.520123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c9c9-3fdd-4f8c-b8ce-51a424884f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:47.520625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c9c9-3fdd-4f8c-b8ce-51a424884f6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:47.524017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:47.526230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:47.529146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:47.529643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:47.530659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:47.531230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:47.533075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:47.545274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9c9c9-3fdd-4f8c-b8ce-51a424884f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7df9c9c9-3fdd-4f8c-b8ce-51a424884f6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:47.550925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:47.551872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:47.552717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:47.555549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 216/421] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:47.557774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 206/422] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:47.562125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:47.567967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9c9c9-3fdd-4f8c-b8ce-51a424884f6a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:47.734834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f06aba-78a1-4fbf-8a52-a8a2640f1d60 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:47.745494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9c9c9-3fdd-4f8c-b8ce-51a424884f6a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:47.768764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:47.782967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:47.783890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:47.788368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:47.788368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:47.788368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:47.788368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:47.788368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:47.797337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 207/423] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:47.803100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:47.834685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df957-15ab-4082-8897-f6e45a74319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:47.835644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df957-15ab-4082-8897-f6e45a74319a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:47.836128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df957-15ab-4082-8897-f6e45a74319a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:47.836651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df957-15ab-4082-8897-f6e45a74319a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:47.932876 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cddbf3-7539-4e8d-bf16-99cf59275612 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:47.973041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9a3db-2ee5-4b02-b66c-6abdd6cbc7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:47.974435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9a3db-2ee5-4b02-b66c-6abdd6cbc7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:47.974920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9a3db-2ee5-4b02-b66c-6abdd6cbc7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:47.975464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9a3db-2ee5-4b02-b66c-6abdd6cbc7d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:47.992094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:48.012018 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8df957-15ab-4082-8897-f6e45a74319a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:48.013711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8df957-15ab-4082-8897-f6e45a74319a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:48.016019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:48.059119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9a3db-2ee5-4b02-b66c-6abdd6cbc7d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:48.060941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9a3db-2ee5-4b02-b66c-6abdd6cbc7d8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:48.182041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 208/424] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:03:48.189456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9a3db-2ee5-4b02-b66c-6abdd6cbc7d8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:48.243805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 217/425] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:48.249463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df957-15ab-4082-8897-f6e45a74319a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:48.274168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:48.349815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9a3db-2ee5-4b02-b66c-6abdd6cbc7d8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:48.368432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:48.371718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df957-15ab-4082-8897-f6e45a74319a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:48.394159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3e0cf-b97e-4528-8383-0be649952e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:48.395090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3e0cf-b97e-4528-8383-0be649952e10 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.395519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3e0cf-b97e-4528-8383-0be649952e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.395939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3e0cf-b97e-4528-8383-0be649952e10 None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.397809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3e0cf-b97e-4528-8383-0be649952e10 None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:48.399864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3e0cf-b97e-4528-8383-0be649952e10 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:48.407198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 209/426] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:48 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:48.408432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72183ab8-e344-472d-ac09-d0e63d1b8f12 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.408736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72183ab8-e344-472d-ac09-d0e63d1b8f12 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.408959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72183ab8-e344-472d-ac09-d0e63d1b8f12 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.410312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 210/427] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:48.410893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72183ab8-e344-472d-ac09-d0e63d1b8f12 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:48.420401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e763e6c-f3af-46aa-9017-06cfeb2668eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:48.421026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763e6c-f3af-46aa-9017-06cfeb2668eb None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.421305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763e6c-f3af-46aa-9017-06cfeb2668eb None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.421573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763e6c-f3af-46aa-9017-06cfeb2668eb None admin] PATH_INFO: `/v3/projects/5d73bd77f96146698091e497e8e76566` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.422702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e763e6c-f3af-46aa-9017-06cfeb2668eb None admin] RBAC: Authorizing `identity:get_project(project_id=5d73bd77f96146698091e497e8e76566)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:48.424197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e763e6c-f3af-46aa-9017-06cfeb2668eb None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:48.426646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 218/428] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:48 2022] GET /identity/v3/projects/5d73bd77f96146698091e497e8e76566 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:48.428243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1c545-37f5-441f-98de-c11f50ee1fec None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.428527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1c545-37f5-441f-98de-c11f50ee1fec None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.428801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1c545-37f5-441f-98de-c11f50ee1fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.454116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4dc1c545-37f5-441f-98de-c11f50ee1fec None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:48.455399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4dc1c545-37f5-441f-98de-c11f50ee1fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:48.542799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 219/429] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:48.543603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1c545-37f5-441f-98de-c11f50ee1fec None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:48.549075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72183ab8-e344-472d-ac09-d0e63d1b8f12 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:48.579448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:48.585894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0ea2f-823d-455d-af91-9ad788ea4b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:48.586526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0ea2f-823d-455d-af91-9ad788ea4b99 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.586809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0ea2f-823d-455d-af91-9ad788ea4b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.587133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0ea2f-823d-455d-af91-9ad788ea4b99 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.588908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0ea2f-823d-455d-af91-9ad788ea4b99 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:48.590462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0ea2f-823d-455d-af91-9ad788ea4b99 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:48.597155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 211/430] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:48 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:48.598212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b8b15-0b69-45da-bcfc-4844d4276ee9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.598500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b8b15-0b69-45da-bcfc-4844d4276ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.598780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b8b15-0b69-45da-bcfc-4844d4276ee9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.600033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 212/431] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:48.600715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b8b15-0b69-45da-bcfc-4844d4276ee9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:48.648272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:48.659331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1c545-37f5-441f-98de-c11f50ee1fec None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:48.694224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998d51c-a965-4830-bc59-fa0fa65721bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:48.694850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998d51c-a965-4830-bc59-fa0fa65721bf None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.695163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998d51c-a965-4830-bc59-fa0fa65721bf None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.695437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998d51c-a965-4830-bc59-fa0fa65721bf None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.696611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998d51c-a965-4830-bc59-fa0fa65721bf None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:48.698882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998d51c-a965-4830-bc59-fa0fa65721bf None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:48.700465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 220/432] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:48 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:48.701217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998d51c-a965-4830-bc59-fa0fa65721bf None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:48.718067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b8b15-0b69-45da-bcfc-4844d4276ee9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:48.755360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62160e2c-c97d-41c7-8206-3194ea74f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:48.756091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160e2c-c97d-41c7-8206-3194ea74f1ac None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.756369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160e2c-c97d-41c7-8206-3194ea74f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.756647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160e2c-c97d-41c7-8206-3194ea74f1ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.767948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62160e2c-c97d-41c7-8206-3194ea74f1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-62160e2c-c97d-41c7-8206-3194ea74f1ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:48.773171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:48.774078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:48.774964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 213/433] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:48.777182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d7ee-7561-4ba8-9481-2e17088767f4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.777182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d7ee-7561-4ba8-9481-2e17088767f4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.777182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46d7ee-7561-4ba8-9481-2e17088767f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.795160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ac46d7ee-7561-4ba8-9481-2e17088767f4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:48.796394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ac46d7ee-7561-4ba8-9481-2e17088767f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:48.812750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998d51c-a965-4830-bc59-fa0fa65721bf None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:48.847169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:48.847773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:48.848295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:48.848295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:48.849036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] RBAC: Adding query filter params (group.id=71f3b4903a4641a9a945faee7df59e99, role.id=347174cd91d94848aec704f00988e871, scope.project.id=5d73bd77f96146698091e497e8e76566) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:48.849447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:48.850883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:48.857209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 221/434] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:48 2022] GET /identity/v3/role_assignments?group.id=71f3b4903a4641a9a945faee7df59e99&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=347174cd91d94848aec704f00988e871 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:48.860246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:48.861333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:48.877999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 214/435] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:48.879298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46d7ee-7561-4ba8-9481-2e17088767f4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:48.896558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:48.982878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f23fc-25cd-4071-a86e-07858cf01c25 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.007222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46d7ee-7561-4ba8-9481-2e17088767f4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.020696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.021379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.021585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.021858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.022633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] RBAC: Adding query filter params (group.id=71f3b4903a4641a9a945faee7df59e99, role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=5d73bd77f96146698091e497e8e76566) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:49.022903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:49.024391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:49.031564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 222/436] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:48 2022] GET /identity/v3/role_assignments?group.id=71f3b4903a4641a9a945faee7df59e99&scope.project.id=5d73bd77f96146698091e497e8e76566&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:49.033176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.049187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.050319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.050798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.051319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.052208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:49.052707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:49.054289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:49.065716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 215/437] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:03:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:49.073632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.151498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:49.158357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:49.176357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8529b-dc97-4254-ac66-59c9786ee3e7 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.205705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba9bd-b55d-465d-9ec8-11bb316b0ade None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.216270 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcb864-4819-4dc1-9cd3-dccbe7f67617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.217064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcb864-4819-4dc1-9cd3-dccbe7f67617 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.217425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcb864-4819-4dc1-9cd3-dccbe7f67617 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.217797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcb864-4819-4dc1-9cd3-dccbe7f67617 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.231372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcb864-4819-4dc1-9cd3-dccbe7f67617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-1afcb864-4819-4dc1-9cd3-dccbe7f67617 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:49.237409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:49.238616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:49.239485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:49.243196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 223/438] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:49.248241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcb864-4819-4dc1-9cd3-dccbe7f67617 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.251087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d3563-8ceb-4dcb-9c89-d09204a2dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.251946 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3563-8ceb-4dcb-9c89-d09204a2dcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.252231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3563-8ceb-4dcb-9c89-d09204a2dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.252508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3563-8ceb-4dcb-9c89-d09204a2dcd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.353331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d3563-8ceb-4dcb-9c89-d09204a2dcd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:49.355503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d3563-8ceb-4dcb-9c89-d09204a2dcd8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:49.383311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcb864-4819-4dc1-9cd3-dccbe7f67617 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.417857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:49.426265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161690e3-d7cf-48ed-9437-a5b7912de909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.426992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161690e3-d7cf-48ed-9437-a5b7912de909 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.427358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161690e3-d7cf-48ed-9437-a5b7912de909 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.427635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161690e3-d7cf-48ed-9437-a5b7912de909 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.432265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161690e3-d7cf-48ed-9437-a5b7912de909 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:49.432265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161690e3-d7cf-48ed-9437-a5b7912de909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:49.432265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161690e3-d7cf-48ed-9437-a5b7912de909 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:49.439780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 224/439] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:49.441013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1576c6-5d31-4152-b64b-af8965d32940 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.441299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1576c6-5d31-4152-b64b-af8965d32940 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.441575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1576c6-5d31-4152-b64b-af8965d32940 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.442775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 225/440] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:49.447920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1576c6-5d31-4152-b64b-af8965d32940 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.460737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:49.475055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 216/441] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:03:49.481526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d3346-50e5-4b2d-b6df-d39388053743 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.481811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d3346-50e5-4b2d-b6df-d39388053743 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.482384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d3346-50e5-4b2d-b6df-d39388053743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.501762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-371d3346-50e5-4b2d-b6df-d39388053743 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:49.503091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-371d3346-50e5-4b2d-b6df-d39388053743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:49.571254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1576c6-5d31-4152-b64b-af8965d32940 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.585930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 217/442] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:49.587072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d3346-50e5-4b2d-b6df-d39388053743 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.617534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcec17-cc84-4124-b5ab-bcfa8f46a7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.618071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcec17-cc84-4124-b5ab-bcfa8f46a7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.618355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcec17-cc84-4124-b5ab-bcfa8f46a7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.618638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcec17-cc84-4124-b5ab-bcfa8f46a7e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.668978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:49.685844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcec17-cc84-4124-b5ab-bcfa8f46a7e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:49.686856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d3346-50e5-4b2d-b6df-d39388053743 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.687333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcec17-cc84-4124-b5ab-bcfa8f46a7e3 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:49.700409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:49.722398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de02d0c-99ee-41d9-b700-f8e73e0d15b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.723059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de02d0c-99ee-41d9-b700-f8e73e0d15b0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.723344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de02d0c-99ee-41d9-b700-f8e73e0d15b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.723624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de02d0c-99ee-41d9-b700-f8e73e0d15b0 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.724743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de02d0c-99ee-41d9-b700-f8e73e0d15b0 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:49.726169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de02d0c-99ee-41d9-b700-f8e73e0d15b0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:49.727800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 218/443] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:49.728689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de02d0c-99ee-41d9-b700-f8e73e0d15b0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.789647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 226/444] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:49.790377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcec17-cc84-4124-b5ab-bcfa8f46a7e3 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.827344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de02d0c-99ee-41d9-b700-f8e73e0d15b0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.861330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97197b-fc5e-46cd-b448-cc3bde7e4e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.861945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97197b-fc5e-46cd-b448-cc3bde7e4e20 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.862227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97197b-fc5e-46cd-b448-cc3bde7e4e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.862500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97197b-fc5e-46cd-b448-cc3bde7e4e20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.873642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97197b-fc5e-46cd-b448-cc3bde7e4e20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5f97197b-fc5e-46cd-b448-cc3bde7e4e20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:49.878310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:49.879272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:49.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 219/445] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:49.880990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97197b-fc5e-46cd-b448-cc3bde7e4e20 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.891486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcec17-cc84-4124-b5ab-bcfa8f46a7e3 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:49.891638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:49.901394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:49.929129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b7f7e-4bbb-4068-8944-6e26d3672920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:49.929879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b7f7e-4bbb-4068-8944-6e26d3672920 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:49.930176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b7f7e-4bbb-4068-8944-6e26d3672920 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:49.930514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b7f7e-4bbb-4068-8944-6e26d3672920 None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:49.932436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b7f7e-4bbb-4068-8944-6e26d3672920 None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:49.933911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b7f7e-4bbb-4068-8944-6e26d3672920 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:49.935519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 227/446] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:49.936418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b7f7e-4bbb-4068-8944-6e26d3672920 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:49.983087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97197b-fc5e-46cd-b448-cc3bde7e4e20 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.018137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad4862-558d-4cd1-864a-08ba5cc222b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:50.018849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4862-558d-4cd1-864a-08ba5cc222b5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:50.019226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4862-558d-4cd1-864a-08ba5cc222b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:50.019505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4862-558d-4cd1-864a-08ba5cc222b5 None admin] PATH_INFO: `/v3/projects/456db541138a40ea9d931140527e9c4a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:50.020622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad4862-558d-4cd1-864a-08ba5cc222b5 None admin] RBAC: Authorizing `identity:get_project(project_id=456db541138a40ea9d931140527e9c4a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:50.022049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad4862-558d-4cd1-864a-08ba5cc222b5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:50.023724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 220/447] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/projects/456db541138a40ea9d931140527e9c4a => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:50.024563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad4862-558d-4cd1-864a-08ba5cc222b5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:50.036059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b7f7e-4bbb-4068-8944-6e26d3672920 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.069844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:50.070444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:50.070723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:50.071005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:50.071718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:50.072029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:50.073451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:50.081462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 228/448] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:50.082248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:50.102019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:50.124081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:50.130570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad4862-558d-4cd1-864a-08ba5cc222b5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.165634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:50.166424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:50.166703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:50.166971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:50.167709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] RBAC: Adding query filter params (group.id=71f3b4903a4641a9a945faee7df59e99, role.id=347174cd91d94848aec704f00988e871, scope.project.id=456db541138a40ea9d931140527e9c4a) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:50.168013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:50.169446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:50.174986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 221/449] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:50 2022] GET /identity/v3/role_assignments?group.id=71f3b4903a4641a9a945faee7df59e99&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=347174cd91d94848aec704f00988e871 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:50.175768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:50.183664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c181b-2a55-4911-90ab-fc2e6dd4300a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.223765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6271b-0857-4052-a736-45f0e83f7d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:50.224423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6271b-0857-4052-a736-45f0e83f7d2a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:50.224703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6271b-0857-4052-a736-45f0e83f7d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:50.224978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6271b-0857-4052-a736-45f0e83f7d2a None admin] PATH_INFO: `/v3/groups/71f3b4903a4641a9a945faee7df59e99` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:50.226207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6271b-0857-4052-a736-45f0e83f7d2a None admin] RBAC: Authorizing `identity:get_group(group_id=71f3b4903a4641a9a945faee7df59e99)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:50.227752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6271b-0857-4052-a736-45f0e83f7d2a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:50.231045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 229/450] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:50 2022] GET /identity/v3/groups/71f3b4903a4641a9a945faee7df59e99 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:50.236463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6271b-0857-4052-a736-45f0e83f7d2a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:50.317279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f2d45-30e5-4f06-9ccc-265f26293d99 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.358853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6271b-0857-4052-a736-45f0e83f7d2a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.361933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3e57-ae90-4be3-bb9e-47277ddf95a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:50.362540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3e57-ae90-4be3-bb9e-47277ddf95a0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:50.362820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3e57-ae90-4be3-bb9e-47277ddf95a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:50.363366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d3e57-ae90-4be3-bb9e-47277ddf95a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:50.366700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:50.377814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:50.401412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:50.406127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:50.406127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:50.406127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:50.406127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] RBAC: Adding query filter params (group.id=71f3b4903a4641a9a945faee7df59e99, role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=456db541138a40ea9d931140527e9c4a) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:50.406127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:50.406127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:50.411471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 230/451] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:50 2022] GET /identity/v3/role_assignments?group.id=71f3b4903a4641a9a945faee7df59e99&scope.project.id=456db541138a40ea9d931140527e9c4a&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:50.454229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d3e57-ae90-4be3-bb9e-47277ddf95a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:50.455730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d3e57-ae90-4be3-bb9e-47277ddf95a0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:50.572439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 222/452] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:50.578108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:50.683052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258294c7-e1c6-465a-bfb4-6b2d9230ad5f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.716799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc6f6-ad3b-4d6a-9a3f-b0e3a607259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:50.717428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc6f6-ad3b-4d6a-9a3f-b0e3a607259f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:50.717710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc6f6-ad3b-4d6a-9a3f-b0e3a607259f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:50.717987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc6f6-ad3b-4d6a-9a3f-b0e3a607259f None admin] PATH_INFO: `/v3/projects/7cc097a340cd405e932b3797d5864d76` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:50.719169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdc6f6-ad3b-4d6a-9a3f-b0e3a607259f None admin] RBAC: Authorizing `identity:get_project(project_id=7cc097a340cd405e932b3797d5864d76)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:50.720625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdc6f6-ad3b-4d6a-9a3f-b0e3a607259f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:50.722543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 231/453] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:03:50 2022] GET /identity/v3/projects/7cc097a340cd405e932b3797d5864d76 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:50.727802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3e57-ae90-4be3-bb9e-47277ddf95a0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:50.769983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:50.825681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d3e57-ae90-4be3-bb9e-47277ddf95a0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.858197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b458b-2204-4a19-9fce-7da11d7f1b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:50.858805 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b458b-2204-4a19-9fce-7da11d7f1b2d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:50.859108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b458b-2204-4a19-9fce-7da11d7f1b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:50.859375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b458b-2204-4a19-9fce-7da11d7f1b2d None admin] PATH_INFO: `/v3/groups/63428a825943497f8c7cc02efa056d05` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:50.860445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b458b-2204-4a19-9fce-7da11d7f1b2d None admin] RBAC: Authorizing `identity:get_group(group_id=63428a825943497f8c7cc02efa056d05)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:50.861846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b458b-2204-4a19-9fce-7da11d7f1b2d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:50.863660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 223/454] 10.222.0.30 () {56 vars in 1180 bytes} [Sun Aug 7 12:03:50 2022] GET /identity/v3/groups/63428a825943497f8c7cc02efa056d05 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:50.869533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc6f6-ad3b-4d6a-9a3f-b0e3a607259f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:50.909064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:50.967718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc6f6-ad3b-4d6a-9a3f-b0e3a607259f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:50.999986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be65aa-ef3d-49ef-8d45-ec7df37f4a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:51.000575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be65aa-ef3d-49ef-8d45-ec7df37f4a39 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:51.000858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be65aa-ef3d-49ef-8d45-ec7df37f4a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:51.001133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be65aa-ef3d-49ef-8d45-ec7df37f4a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:51.001850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be65aa-ef3d-49ef-8d45-ec7df37f4a39 None admin] RBAC: Adding query filter params (group.id=63428a825943497f8c7cc02efa056d05, role.id=f72d3388f13149b483355331380f040b, scope.project.id=7cc097a340cd405e932b3797d5864d76) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:51.002160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be65aa-ef3d-49ef-8d45-ec7df37f4a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:51.003628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be65aa-ef3d-49ef-8d45-ec7df37f4a39 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:51.009150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 232/455] 10.222.0.30 () {56 vars in 1376 bytes} [Sun Aug 7 12:03:50 2022] GET /identity/v3/role_assignments?group.id=63428a825943497f8c7cc02efa056d05&scope.project.id=7cc097a340cd405e932b3797d5864d76&role.id=f72d3388f13149b483355331380f040b => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:53.560158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba5ed9-d980-4b8a-92f4-c1321b3bb1c2 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:53.561181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba5ed9-d980-4b8a-92f4-c1321b3bb1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:53.561739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba5ed9-d980-4b8a-92f4-c1321b3bb1c2 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:53.563556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 224/456] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:53.568830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbcbf1-ed77-4d9d-984c-099bea6f8e5f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:53.569672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbcbf1-ed77-4d9d-984c-099bea6f8e5f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:53.570142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbcbf1-ed77-4d9d-984c-099bea6f8e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:53.589731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-83fbcbf1-ed77-4d9d-984c-099bea6f8e5f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:53.591364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-83fbcbf1-ed77-4d9d-984c-099bea6f8e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:53.642505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c86ed-ae62-4ce1-8b77-b730e4280567 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:53.642812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c86ed-ae62-4ce1-8b77-b730e4280567 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:53.643166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c86ed-ae62-4ce1-8b77-b730e4280567 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:53.644624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 225/457] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:53.649672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7058bf5-df93-4eaa-ad8b-65cf2f40ed84 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:53.649955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7058bf5-df93-4eaa-ad8b-65cf2f40ed84 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:53.650762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7058bf5-df93-4eaa-ad8b-65cf2f40ed84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:53.685341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d7058bf5-df93-4eaa-ad8b-65cf2f40ed84 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:53.686903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 233/458] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:53.688371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d7058bf5-df93-4eaa-ad8b-65cf2f40ed84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:53.728677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbcbf1-ed77-4d9d-984c-099bea6f8e5f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:53.777187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:53.797678 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 226/459] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:53.840530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7058bf5-df93-4eaa-ad8b-65cf2f40ed84 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:53.896213 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbcbf1-ed77-4d9d-984c-099bea6f8e5f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:53.912286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:53.943973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:53.944960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:53.945451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:53.945950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:53.977354 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:53.980004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:54.019255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:54.020290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:54.021216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:54.021216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:54.021216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:54.024662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 234/460] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:54.032283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:54.041537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7058bf5-df93-4eaa-ad8b-65cf2f40ed84 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:54.078203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d935a5a-c237-4c3f-bca8-cb604cbbc1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:54.078840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d935a5a-c237-4c3f-bca8-cb604cbbc1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.079149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d935a5a-c237-4c3f-bca8-cb604cbbc1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.079424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d935a5a-c237-4c3f-bca8-cb604cbbc1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.090097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d935a5a-c237-4c3f-bca8-cb604cbbc1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:54.091683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d935a5a-c237-4c3f-bca8-cb604cbbc1f8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3d935a5a-c237-4c3f-bca8-cb604cbbc1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:54.107522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:54.108513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:54.109382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:54.109382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:54.109382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:54.113808 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 227/461] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:54.115070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144e854-3207-4d07-b557-3a35881a2610 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.115362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144e854-3207-4d07-b557-3a35881a2610 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.115642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144e854-3207-4d07-b557-3a35881a2610 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.120578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 228/462] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:54.123052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b911-d85b-4d21-aca8-0e3850c9c234 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.123345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b911-d85b-4d21-aca8-0e3850c9c234 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.123623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b911-d85b-4d21-aca8-0e3850c9c234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.134226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:54.142896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e921b911-d85b-4d21-aca8-0e3850c9c234 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:54.146847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e921b911-d85b-4d21-aca8-0e3850c9c234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:54.151974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a976f3-6a2d-4dc5-a8a7-975292319d5f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:54.188496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:54.189557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.190040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.190513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.191389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:54.191895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:54.193544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:54.216947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 235/463] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:54.216947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:54.218634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 229/464] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:54.227924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921b911-d85b-4d21-aca8-0e3850c9c234 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:54.258907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:54.328644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f3286-386d-4147-822d-3d44ee519573 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:54.357084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921b911-d85b-4d21-aca8-0e3850c9c234 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:54.364389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:54.364995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.365273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.365543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.366183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:54.366482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:54.367947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:54.379655 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 236/465] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:54.384283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:54.387104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:54.401447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1406b-d690-4a9f-aae6-6577fbf96132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:54.402071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1406b-d690-4a9f-aae6-6577fbf96132 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.402345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1406b-d690-4a9f-aae6-6577fbf96132 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.402611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1406b-d690-4a9f-aae6-6577fbf96132 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.403267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1406b-d690-4a9f-aae6-6577fbf96132 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:54.404671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1406b-d690-4a9f-aae6-6577fbf96132 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:54.482612 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad5ad7-4a37-4dc1-8d6f-d0aa980ccd37 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:54.510639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 230/466] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:54 2022] POST /identity/v3/users => generated 246 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:54.511902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0b134-db0f-4e27-83d4-7022c0305d06 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.512185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0b134-db0f-4e27-83d4-7022c0305d06 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.512459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0b134-db0f-4e27-83d4-7022c0305d06 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.513961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 231/467] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:54.515858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0b134-db0f-4e27-83d4-7022c0305d06 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:54.529726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071dd48-5d03-4c9c-a4a7-40b974d6d314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:54.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071dd48-5d03-4c9c-a4a7-40b974d6d314 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.531147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071dd48-5d03-4c9c-a4a7-40b974d6d314 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.531568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071dd48-5d03-4c9c-a4a7-40b974d6d314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.545447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071dd48-5d03-4c9c-a4a7-40b974d6d314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:54.548110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071dd48-5d03-4c9c-a4a7-40b974d6d314 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-7071dd48-5d03-4c9c-a4a7-40b974d6d314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:54.564391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:54.565408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:54.566408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:54.566408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:54.566408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:54.568015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 237/468] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:54.569543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db7809-412c-4b4f-a30f-87b8ad310f23 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.570492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db7809-412c-4b4f-a30f-87b8ad310f23 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.570919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db7809-412c-4b4f-a30f-87b8ad310f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.593546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-23db7809-412c-4b4f-a30f-87b8ad310f23 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:54.595271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-23db7809-412c-4b4f-a30f-87b8ad310f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:54.647878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:54.668960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:54.686086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0b134-db0f-4e27-83d4-7022c0305d06 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:54.690768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 238/469] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:54.691994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db7809-412c-4b4f-a30f-87b8ad310f23 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:54.738486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15598bc-3191-41f6-b358-38cf3563f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:54.739616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15598bc-3191-41f6-b358-38cf3563f4f0 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.740127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15598bc-3191-41f6-b358-38cf3563f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.740896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15598bc-3191-41f6-b358-38cf3563f4f0 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.741546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15598bc-3191-41f6-b358-38cf3563f4f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:54.742986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15598bc-3191-41f6-b358-38cf3563f4f0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:54.812939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db7809-412c-4b4f-a30f-87b8ad310f23 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:54.847359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:54.847978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:54.848308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:54.848627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:54.849266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:54.849563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:54.850949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:54.868336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 232/470] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:54 2022] POST /identity/v3/users => generated 248 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:54.870228 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15598bc-3191-41f6-b358-38cf3563f4f0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:54.870653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 239/471] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:54.878192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:54.943845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:55.007705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965007f3-edd4-4b7c-8b67-b97714799944 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:55.013581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15598bc-3191-41f6-b358-38cf3563f4f0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:55.047187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:55.047504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cc6a9-8fee-4d24-8adc-d9c52ab67e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:55.048321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cc6a9-8fee-4d24-8adc-d9c52ab67e55 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:55.048321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cc6a9-8fee-4d24-8adc-d9c52ab67e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:55.048592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cc6a9-8fee-4d24-8adc-d9c52ab67e55 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:55.049261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cc6a9-8fee-4d24-8adc-d9c52ab67e55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:55.050715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cc6a9-8fee-4d24-8adc-d9c52ab67e55 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:55.052766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b2d27-5b50-4105-95d1-3314702f2fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:55.052766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b2d27-5b50-4105-95d1-3314702f2fcb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:55.053048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b2d27-5b50-4105-95d1-3314702f2fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:55.053267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b2d27-5b50-4105-95d1-3314702f2fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:55.068127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b2d27-5b50-4105-95d1-3314702f2fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:55.069705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b2d27-5b50-4105-95d1-3314702f2fcb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c00b2d27-5b50-4105-95d1-3314702f2fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:55.086410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:55.087428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:55.089484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:55.089484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:55.089484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:55.089484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 233/472] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:55.089712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e077-081e-47c1-a292-e1bc842752aa None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:55.089937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e077-081e-47c1-a292-e1bc842752aa None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:55.090186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e077-081e-47c1-a292-e1bc842752aa None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:55.091478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 234/473] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:55.095847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72e077-081e-47c1-a292-e1bc842752aa None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:55.185618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 240/474] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:54 2022] POST /identity/v3/users => generated 248 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:55.195864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa8de-b2e3-4b1b-a472-d6b7e5710b1b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:55.196055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa8de-b2e3-4b1b-a472-d6b7e5710b1b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:55.196342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa8de-b2e3-4b1b-a472-d6b7e5710b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:55.207829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72e077-081e-47c1-a292-e1bc842752aa None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:55.225938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0e0fa8de-b2e3-4b1b-a472-d6b7e5710b1b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:55.227232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0e0fa8de-b2e3-4b1b-a472-d6b7e5710b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:55.246350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:55.247167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:55.247545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:55.247917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:55.248694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:55.249107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:55.250701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:55.264992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 235/475] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:55.270797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:55.377918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:55.379222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 241/476] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:55.417763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fa8de-b2e3-4b1b-a472-d6b7e5710b1b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:55.452072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c74349-e95f-4bfd-8ef2-046710d94746 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:55.458945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:55.491590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ab992-21b3-4c79-b568-d8a2dae02d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:55.492229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab992-21b3-4c79-b568-d8a2dae02d3a None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:55.492506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab992-21b3-4c79-b568-d8a2dae02d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:55.492777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab992-21b3-4c79-b568-d8a2dae02d3a None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:55.493419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ab992-21b3-4c79-b568-d8a2dae02d3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:55.494874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ab992-21b3-4c79-b568-d8a2dae02d3a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:55.599489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fa8de-b2e3-4b1b-a472-d6b7e5710b1b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:55.638787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:55.639428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:55.639710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:55.639995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:55.640699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:55.642077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:55.662039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 236/477] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:03:55 2022] POST /identity/v3/users => generated 249 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:03:55.673010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.sql.core [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 12:03:55.673010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 07 12:03:55.673010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [parameters: {'id': 'c0d2ec1b4d1f4937aafbf179e2d40906', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 07 12:03:55.673010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80432) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application conn.query(q) Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result.read() Aug 07 12:03:55.694344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return ref.to_dict() Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 12:03:55.695346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application next(self.gen) Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application yield resource Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application next(self.gen) Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise self.value Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application session.commit() Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 12:03:55.696290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.session.flush() Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application compat.raise_( Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 12:03:55.698523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 12:03:55.700676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application conn.query(q) Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result.read() Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 12:03:55.702710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [parameters: {'id': 'c0d2ec1b4d1f4937aafbf179e2d40906', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:55.704662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 07 12:03:55.711505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:55.713594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 242/478] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 12:03:55 2022] POST /identity/v3/roles => generated 144 bytes in 279 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:55.713739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ab992-21b3-4c79-b568-d8a2dae02d3a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:55.852324 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:55.855738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ab992-21b3-4c79-b568-d8a2dae02d3a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:55.901621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8b2c9-5fcd-4d78-844c-a2322ba1547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:55.902321 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8b2c9-5fcd-4d78-844c-a2322ba1547f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:55.902605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8b2c9-5fcd-4d78-844c-a2322ba1547f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:55.902879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8b2c9-5fcd-4d78-844c-a2322ba1547f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:55.913525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8b2c9-5fcd-4d78-844c-a2322ba1547f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-14c8b2c9-5fcd-4d78-844c-a2322ba1547f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:55.918387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:55.920456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:55.922267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 237/479] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:03:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:55.925456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:56.028177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:56.034544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87178276-a2e2-47d4-a199-b9fef04dcefc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:56.070595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae65127-4134-44e8-8b78-8f27b82fff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:56.071255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65127-4134-44e8-8b78-8f27b82fff53 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:56.071534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65127-4134-44e8-8b78-8f27b82fff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:56.071807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65127-4134-44e8-8b78-8f27b82fff53 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:56.073079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65127-4134-44e8-8b78-8f27b82fff53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:56.073382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65127-4134-44e8-8b78-8f27b82fff53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:56.076228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65127-4134-44e8-8b78-8f27b82fff53 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:56.085188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 243/480] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:03:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:03:56.879178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3917-e381-4aa5-9c34-912f9fb8df42 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:56.879699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3917-e381-4aa5-9c34-912f9fb8df42 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:56.879699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3917-e381-4aa5-9c34-912f9fb8df42 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:56.881452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 238/481] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:56.886958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbe215-4069-4fc5-8c9e-75b7352a48b5 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:56.887416 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbe215-4069-4fc5-8c9e-75b7352a48b5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:56.887510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbe215-4069-4fc5-8c9e-75b7352a48b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:56.916637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7ebbe215-4069-4fc5-8c9e-75b7352a48b5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:56.917898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7ebbe215-4069-4fc5-8c9e-75b7352a48b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:57.013773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 244/482] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:57.042650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3917-e381-4aa5-9c34-912f9fb8df42 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:57.193128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3917-e381-4aa5-9c34-912f9fb8df42 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:57.244318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099ccbe-dd50-44e5-ad44-d256647d703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:57.244956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099ccbe-dd50-44e5-ad44-d256647d703d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.245229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099ccbe-dd50-44e5-ad44-d256647d703d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.245495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099ccbe-dd50-44e5-ad44-d256647d703d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.263915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099ccbe-dd50-44e5-ad44-d256647d703d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a099ccbe-dd50-44e5-ad44-d256647d703d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:57.269016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:57.269986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:57.270831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 239/483] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:03:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:57.276653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbe215-4069-4fc5-8c9e-75b7352a48b5 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:57.357769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5b530-103c-46af-ad8a-b5766db068f7 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.358063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5b530-103c-46af-ad8a-b5766db068f7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.358323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5b530-103c-46af-ad8a-b5766db068f7 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.359912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 240/484] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:03:57.369916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19811c-0aa3-4de4-8aa3-4e429f33bfe4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.370198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19811c-0aa3-4de4-8aa3-4e429f33bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.370471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19811c-0aa3-4de4-8aa3-4e429f33bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.407051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4c19811c-0aa3-4de4-8aa3-4e429f33bfe4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:57.407051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4c19811c-0aa3-4de4-8aa3-4e429f33bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:57.427982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:57.440462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:57.441458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbe215-4069-4fc5-8c9e-75b7352a48b5 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:57.479073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:57.480768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.481332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.481773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.482604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:57.486079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:57.486079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:57.494941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 245/485] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:03:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:57.500886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:57.535644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 241/486] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:03:57.559034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19811c-0aa3-4de4-8aa3-4e429f33bfe4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:57.673379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19811c-0aa3-4de4-8aa3-4e429f33bfe4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:57.710224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4c4a7-841d-47cd-a993-6897ee4373ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:57.710854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4c4a7-841d-47cd-a993-6897ee4373ff None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.711167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4c4a7-841d-47cd-a993-6897ee4373ff None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.711436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4c4a7-841d-47cd-a993-6897ee4373ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.722653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4c4a7-841d-47cd-a993-6897ee4373ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7fa4c4a7-841d-47cd-a993-6897ee4373ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:57.727357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:57.728333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:57.729198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 242/487] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:03:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:57.744302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4c4a7-841d-47cd-a993-6897ee4373ff None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:57.763383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:57.771520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70734c-de09-4f93-944f-13eff101448f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:57.820299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b7a2e-6a51-4c2f-a343-d24ba227f167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:57.821264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b7a2e-6a51-4c2f-a343-d24ba227f167 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.821732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b7a2e-6a51-4c2f-a343-d24ba227f167 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.822904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b7a2e-6a51-4c2f-a343-d24ba227f167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.836856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b7a2e-6a51-4c2f-a343-d24ba227f167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:57.838671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b7a2e-6a51-4c2f-a343-d24ba227f167 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-506b7a2e-6a51-4c2f-a343-d24ba227f167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:57.854902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:57.855937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:57.856821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:57.856821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:57.856821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:57.862884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 246/488] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:57.864694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972533c-a115-4c01-947f-79884353bfa7 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.865129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972533c-a115-4c01-947f-79884353bfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.865572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972533c-a115-4c01-947f-79884353bfa7 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.867216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 247/489] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:57.868104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972533c-a115-4c01-947f-79884353bfa7 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:57.875395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:57.877898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4c4a7-841d-47cd-a993-6897ee4373ff None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:57.914638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b6c84-9f4e-4440-bda1-40153a8dddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:57.915308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b6c84-9f4e-4440-bda1-40153a8dddb4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.915470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b6c84-9f4e-4440-bda1-40153a8dddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.916319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b6c84-9f4e-4440-bda1-40153a8dddb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.916990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b6c84-9f4e-4440-bda1-40153a8dddb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:57.917289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b6c84-9f4e-4440-bda1-40153a8dddb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:57.918663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b6c84-9f4e-4440-bda1-40153a8dddb4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:57.931642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 243/490] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:03:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:57.931642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1271b30-34f2-4736-851d-08d1b36f9a5f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:57.931642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1271b30-34f2-4736-851d-08d1b36f9a5f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:57.931642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1271b30-34f2-4736-851d-08d1b36f9a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:57.948916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b1271b30-34f2-4736-851d-08d1b36f9a5f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:57.950110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b1271b30-34f2-4736-851d-08d1b36f9a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:57.995623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972533c-a115-4c01-947f-79884353bfa7 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.026074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 244/491] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:58.026074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1271b30-34f2-4736-851d-08d1b36f9a5f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.026670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:58.039107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.041015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.041015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.041015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.041929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:58.042390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.044123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.058297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 248/492] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:58.065189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.137996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:58.157047 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1271b30-34f2-4736-851d-08d1b36f9a5f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.183034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0710f-bc4a-4d58-9a6c-81973732399c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.194419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.195305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.195754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.196169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.207242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.208978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.221448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.222080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.222362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.222639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:58.225396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:58.226490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:58.227569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:58.227569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:58.227569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:58.229380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 245/493] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:58.230313 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.235431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.236916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:03:58.252387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:58.253541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:03:58.254523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:58.258478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 249/494] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:03:58.259527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.293198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:58.339838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b17a67-0e08-4ef8-8b94-d3d6068d5ac3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.371858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d308c2d-21fb-4de6-8820-df45520a1439 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.376238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41c08-23ed-4414-95ab-6f860facaaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.376868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41c08-23ed-4414-95ab-6f860facaaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.377153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41c08-23ed-4414-95ab-6f860facaaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.377429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41c08-23ed-4414-95ab-6f860facaaf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.389417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee41c08-23ed-4414-95ab-6f860facaaf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-0ee41c08-23ed-4414-95ab-6f860facaaf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:58.394528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:58.395473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:03:58.396289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:58.398362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:58.400567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 246/495] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:58.401525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41c08-23ed-4414-95ab-6f860facaaf0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.411314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a0757-c301-4299-933a-95d1988e8977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.411959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a0757-c301-4299-933a-95d1988e8977 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.412609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a0757-c301-4299-933a-95d1988e8977 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.412890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a0757-c301-4299-933a-95d1988e8977 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.413549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a0757-c301-4299-933a-95d1988e8977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:58.414136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a0757-c301-4299-933a-95d1988e8977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.416423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a0757-c301-4299-933a-95d1988e8977 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.437223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 250/496] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:58.437223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f4c1c-5ff5-4b74-be41-b06621368ffb None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.437223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f4c1c-5ff5-4b74-be41-b06621368ffb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.437223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f4c1c-5ff5-4b74-be41-b06621368ffb None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.439965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 251/497] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:58.439965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f4c1c-5ff5-4b74-be41-b06621368ffb None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.513911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41c08-23ed-4414-95ab-6f860facaaf0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.529651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:58.552095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.552628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f4c1c-5ff5-4b74-be41-b06621368ffb None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.552868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.553223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.553573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.554379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:58.554687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.556159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.589059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a276359-88e4-49a4-ae5b-8ac2d31ae574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.589658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a276359-88e4-49a4-ae5b-8ac2d31ae574 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.589932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a276359-88e4-49a4-ae5b-8ac2d31ae574 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.590199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a276359-88e4-49a4-ae5b-8ac2d31ae574 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.590857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a276359-88e4-49a4-ae5b-8ac2d31ae574 None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:58.591192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a276359-88e4-49a4-ae5b-8ac2d31ae574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.592567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a276359-88e4-49a4-ae5b-8ac2d31ae574 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.597404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 247/498] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:58.598972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.602182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 252/499] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:58.604648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f557122-b519-4496-8628-a7501bd3e4a8 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.604937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f557122-b519-4496-8628-a7501bd3e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.605217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f557122-b519-4496-8628-a7501bd3e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.623589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6f557122-b519-4496-8628-a7501bd3e4a8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:58.624783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6f557122-b519-4496-8628-a7501bd3e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:58.662724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:58.690774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 253/500] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:58.699714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aee31-c300-473d-9b04-97ff44df60e0 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.699714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aee31-c300-473d-9b04-97ff44df60e0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.699714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aee31-c300-473d-9b04-97ff44df60e0 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.699714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 254/501] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:03:58.699714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396aee31-c300-473d-9b04-97ff44df60e0 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.702247 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6019b-593a-46c5-8be1-50fcd7278f35 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.735922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.736547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.736827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.737093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.744872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.746303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:03:58.758982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:58.760117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:58.761413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 248/502] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:58.763869 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:58.795608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:58.800452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396aee31-c300-473d-9b04-97ff44df60e0 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.833833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296d1a7-4d08-47ea-9e7d-fc0a33589a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.834425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296d1a7-4d08-47ea-9e7d-fc0a33589a0a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.834704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296d1a7-4d08-47ea-9e7d-fc0a33589a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.834976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296d1a7-4d08-47ea-9e7d-fc0a33589a0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.861818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3fb6-16bc-4a44-a464-dd53702e46c2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:58.877091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:58.895160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc97940-0487-4c3b-a57f-e8f44fcc0789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:58.895772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc97940-0487-4c3b-a57f-e8f44fcc0789 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.896067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc97940-0487-4c3b-a57f-e8f44fcc0789 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.896314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc97940-0487-4c3b-a57f-e8f44fcc0789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.902190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296d1a7-4d08-47ea-9e7d-fc0a33589a0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.904130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296d1a7-4d08-47ea-9e7d-fc0a33589a0a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.906977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc97940-0487-4c3b-a57f-e8f44fcc0789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:58.908430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc97940-0487-4c3b-a57f-e8f44fcc0789 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9dc97940-0487-4c3b-a57f-e8f44fcc0789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:58.922663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:58.924151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:58.925233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:58.925233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:58.925233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:58.925233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 249/503] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:58.925233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3bad6-c129-4693-af09-f4ef29dd769f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:58.925233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3bad6-c129-4693-af09-f4ef29dd769f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:58.925600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3bad6-c129-4693-af09-f4ef29dd769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:58.943265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d2c3bad6-c129-4693-af09-f4ef29dd769f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:03:58.944446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d2c3bad6-c129-4693-af09-f4ef29dd769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:03:59.002858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 255/504] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:59.004398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296d1a7-4d08-47ea-9e7d-fc0a33589a0a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.014297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 250/505] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:03:59.015147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3bad6-c129-4693-af09-f4ef29dd769f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.040658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.078161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.101701 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296d1a7-4d08-47ea-9e7d-fc0a33589a0a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.130945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3bad6-c129-4693-af09-f4ef29dd769f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.137267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fded7d-6f1b-41ea-9293-e934cd1e9a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.137890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fded7d-6f1b-41ea-9293-e934cd1e9a89 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.138174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fded7d-6f1b-41ea-9293-e934cd1e9a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.138444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fded7d-6f1b-41ea-9293-e934cd1e9a89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.148309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fded7d-6f1b-41ea-9293-e934cd1e9a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-99fded7d-6f1b-41ea-9293-e934cd1e9a89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:03:59.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:59.153927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:59.154767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 256/506] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:59.155476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fded7d-6f1b-41ea-9293-e934cd1e9a89 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.167652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.168246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.168519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.168791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.169509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:59.169808 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.171238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.189687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 251/507] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:59.190458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.257030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fded7d-6f1b-41ea-9293-e934cd1e9a89 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.258276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.285936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55378cec-f128-4704-99a6-83fcd1f8d840 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.290187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.291446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.291726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.291995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.292632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:59.292933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.294313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.295809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.305835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 257/508] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:59.312497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.324505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.325126 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.325400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.325669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.335916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.337289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:59.351614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:59.352582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:59.354075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:59.354075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:59.354075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:59.354075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 252/509] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:59.354940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.409054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8084f-4480-453a-9947-6c43fa96d6ea None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.442895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.443556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.443836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.444108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.454106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dcb43-2328-45b1-9ec6-8f9a2aed46cd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.454558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.456032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:59.469864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:59.470803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:59.471727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:59.471727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:59.471727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:03:59.471727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 258/510] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:59.471955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.480131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.492410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.493007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.493289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.493567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.494929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:59.495261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.496617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.504944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 253/511] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:59.505735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.523355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.577236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7c8c7-dcb2-4a8c-961f-0f3327159247 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.604453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c189ebe-1817-4abf-81c5-c7bc6dd84199 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.613157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c416b3-04f5-464b-8dd5-0c1301585018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.614005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c416b3-04f5-464b-8dd5-0c1301585018 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.614281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c416b3-04f5-464b-8dd5-0c1301585018 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.614551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c416b3-04f5-464b-8dd5-0c1301585018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.639502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.640121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.640400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.640667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.647428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.648858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:03:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:59.660947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 254/512] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:59.661867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.685476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c416b3-04f5-464b-8dd5-0c1301585018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.686951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c416b3-04f5-464b-8dd5-0c1301585018 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.709280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.733915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.760723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e830bb9-e1e3-4211-ad96-59052f14be0d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.787080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 259/513] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:03:59.787899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c416b3-04f5-464b-8dd5-0c1301585018 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.800197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.800800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.801081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.801355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.801995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:59.802296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.803687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.816128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 255/514] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:59.817002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.884945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c416b3-04f5-464b-8dd5-0c1301585018 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.914239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e654202-34d5-4cc9-b4dd-9f6d32314089 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:03:59.918720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.919344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.919626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.919905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.920541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:03:59.920842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.922230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.922559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.934155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 260/515] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:03:59.936611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:03:59.945628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:03:59.950092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:03:59.950717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:03:59.950993 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:03:59.951289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:03:59.961554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:03:59.963071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:03:59.977271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:03:59.978795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:03:59.979829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:03:59.979829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:03:59.979829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:03:59.979829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 256/516] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:03:59.980134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.042313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae01ba-1ed4-4fa2-bfeb-75bebce9f8b0 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.074588 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.075202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.075477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.075742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.076453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:00.076749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.077014 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184018-8b5d-4124-be63-5b35c2243911 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.078165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.112834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.113712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.113993 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.114266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.115116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 261/517] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:00.118081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.129147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.130588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:00.145388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:00.146318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:00.147255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:00.147255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:00.147255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:00.147255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 257/518] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:00.147255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.160383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:00.182135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:00.218939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9987b29-3494-448a-923d-792ad8b62054 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.248353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9ac15-2490-45d9-a876-f1a0b8ff214d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.253061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.253685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.253965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.254238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.260901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.262353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:00.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:00.275790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 262/519] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:00.276965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.285340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384b6e0-2564-497c-b1da-a30181678986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.285939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b6e0-2564-497c-b1da-a30181678986 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.286221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b6e0-2564-497c-b1da-a30181678986 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.286495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b6e0-2564-497c-b1da-a30181678986 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.287181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384b6e0-2564-497c-b1da-a30181678986 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.288605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384b6e0-2564-497c-b1da-a30181678986 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.372870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103cd82-ed3b-401f-931c-711afeda910f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.375792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 258/520] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:04:00 2022] POST /identity/v3/users => generated 247 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:00.376479 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384b6e0-2564-497c-b1da-a30181678986 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.406169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:00.418500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.419145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.419419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.419752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.420404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:00.420703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.422203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.441544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 263/521] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:00.441544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.463802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:00.503095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384b6e0-2564-497c-b1da-a30181678986 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.544415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b4b1-88ea-4776-b141-829629d1d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.545039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b4b1-88ea-4776-b141-829629d1d26b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.545352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b4b1-88ea-4776-b141-829629d1d26b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.545629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b4b1-88ea-4776-b141-829629d1d26b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.559120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b217b3-fdc5-49cf-8102-7ade59b5f554 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.600558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.601162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.601443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.601715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.602365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:00.602664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.604371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.617007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 264/522] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:00.617801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.618970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b4b1-88ea-4776-b141-829629d1d26b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.620593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b4b1-88ea-4776-b141-829629d1d26b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.666804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:00.688765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:00.722661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17986f-0b85-4881-93bc-035945a7e03f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.735165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 259/523] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:04:00.735975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b4b1-88ea-4776-b141-829629d1d26b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.758620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.759258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.759569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.759845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.760533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:00.760838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.762256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.785016 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 265/524] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:00.785016 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.850464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b4b1-88ea-4776-b141-829629d1d26b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.887044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.887732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.887836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc486-f4e4-49b9-9670-2f7c4f08dad1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:00.887999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.888266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.891041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:00.896159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.897583 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:00.909374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:00.916236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 260/525] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:00.917326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:00.929975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:00.930603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:00.930880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:00.931350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:00.938826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:00.940224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:00.941318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:00.952268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:00.953166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 266/526] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:00.954681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:01.023703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d549-c9b1-4fef-a05f-17a673ee6b85 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.057387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5307a-a745-4284-a680-c5e73ec4bc5b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.060564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:01.061189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:01.061463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:01.061727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:01.072667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:01.074207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:01.090341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:01.091486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:01.092355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:01.092355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:01.092355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:01.092355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 261/527] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:01.092355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:01.097339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:01.097949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:01.098229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:01.098503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:01.099281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=cb2075e9fff3459881c13f81456ab198) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:01.099586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:01.101019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:01.111243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 267/528] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:00 2022] GET /identity/v3/role_assignments?user.id=cb2075e9fff3459881c13f81456ab198&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:01.112252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:01.152853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:01.197196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:01.230350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944f773-d674-4e8e-9a6c-a6d16fdd7242 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.254696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623a3b7-35c9-46f7-a003-42ea5e514031 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.280340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c8857-80c6-4750-8345-690161492229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:01.280953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c8857-80c6-4750-8345-690161492229 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:01.281230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c8857-80c6-4750-8345-690161492229 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:01.281510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c8857-80c6-4750-8345-690161492229 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:01.282478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c8857-80c6-4750-8345-690161492229 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:01.282786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c8857-80c6-4750-8345-690161492229 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:01.284664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c8857-80c6-4750-8345-690161492229 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:01.296915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:01.297914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:01.298375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:01.298827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:01.299825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:01.300312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:01.301927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:01.311085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 262/529] 10.222.0.30 () {56 vars in 1139 bytes} [Sun Aug 7 12:04:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:01.312250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c8857-80c6-4750-8345-690161492229 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:01.318274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 268/530] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:01.319146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:01.441553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:01.457119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c8857-80c6-4750-8345-690161492229 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.458532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6dd3f-e575-49c3-8780-1fc337ffd8dc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.493007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:01.495033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:01.495033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:01.495033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:01.495033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:01.495375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:01.496672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:01.499895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:01.505355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86395b9-ab4c-46f6-963f-4c329b592694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:01.506636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86395b9-ab4c-46f6-963f-4c329b592694 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:01.507376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86395b9-ab4c-46f6-963f-4c329b592694 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:01.507931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86395b9-ab4c-46f6-963f-4c329b592694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:01.514575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 263/531] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:01.514575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:01.607114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86395b9-ab4c-46f6-963f-4c329b592694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:01.608900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86395b9-ab4c-46f6-963f-4c329b592694 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:01.636206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f705be-4bf9-429e-abce-1f0e3a8a0119 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.672990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:01.673785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:01.674160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:01.674511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:01.675467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=6b5ee9e4388d4bbabc7f1e9e3ca50b67) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:01.675847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:01.677424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:01.684459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 264/532] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:01 2022] GET /identity/v3/role_assignments?user.id=6b5ee9e4388d4bbabc7f1e9e3ca50b67&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:01.685466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:01.742242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:01.769961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 269/533] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:04:01.777338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86395b9-ab4c-46f6-963f-4c329b592694 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:01.804422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:01.867165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d197f-d53f-4f53-99a9-4324d89b7431 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.936813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:01.937447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:01.937721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:01.937987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:01.947471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:01.948901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:01.966226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:01.967171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:01.968461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 265/534] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:01.969264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86395b9-ab4c-46f6-963f-4c329b592694 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:01.973089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:02.080377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb414c-501a-4bfe-be1c-f5dbdaa8e8d8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:02.080985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:02.081335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.081572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.081721 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.102734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:02.106714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:02.123027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:02.136247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:02.138307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:02.139218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:02.139218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:02.139218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:02.144581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 270/535] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:02.145478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:02.151289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:02.151893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.152203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.152503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.153198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:02.153496 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:02.154906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:02.175151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 266/536] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:02.181295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:02.233608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:02.300769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afdb74-12f0-4457-8dc9-f775bf26a72d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:02.342772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:02.343313 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.343590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.344331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.345061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=e0792940b93b4c1998be971f6143a514) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:02.345365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:02.346782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:02.353969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 267/537] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:02 2022] GET /identity/v3/role_assignments?user.id=e0792940b93b4c1998be971f6143a514&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:02.358151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edab4a-2a30-48cf-9cbd-8c25162bda55 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:02.422778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e5df8-64da-41f0-a3c7-f54626fbf046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:02.423505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e5df8-64da-41f0-a3c7-f54626fbf046 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.423791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e5df8-64da-41f0-a3c7-f54626fbf046 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.424065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e5df8-64da-41f0-a3c7-f54626fbf046 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.424736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e5df8-64da-41f0-a3c7-f54626fbf046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:02.425044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e5df8-64da-41f0-a3c7-f54626fbf046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:02.426462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e5df8-64da-41f0-a3c7-f54626fbf046 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:02.445860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 271/538] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:02.451716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:02.512641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed275e33-8e54-44b7-8009-788067e9cfc2 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.513292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed275e33-8e54-44b7-8009-788067e9cfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.513775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed275e33-8e54-44b7-8009-788067e9cfc2 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.516806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 272/539] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:02.521427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b1607-43cd-47df-b62f-1586afe1ba3c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.521959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b1607-43cd-47df-b62f-1586afe1ba3c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.522440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b1607-43cd-47df-b62f-1586afe1ba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.533581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:02.561176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-756b1607-43cd-47df-b62f-1586afe1ba3c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:02.565133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-756b1607-43cd-47df-b62f-1586afe1ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:02.622543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3088f08-6cd7-499c-bd38-faa3398b2896 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:02.690708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 273/540] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:04:02.692965 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:02.693842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.694334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.694800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.702100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:02.703974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:02.716266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:02.717169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:02.718043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:02.718043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:02.718043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:02.718043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:02.718043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:02.718043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:02.718043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:02.718043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:02.720383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 268/541] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:02.726202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b1607-43cd-47df-b62f-1586afe1ba3c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:02.734125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:02.760877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:02.858176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b1607-43cd-47df-b62f-1586afe1ba3c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:02.871139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec479a-61dd-4907-b018-6086f4d17693 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:02.895872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:02.917929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96216424-e577-48a5-9a68-925dc5142adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:02.918572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216424-e577-48a5-9a68-925dc5142adc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.918849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216424-e577-48a5-9a68-925dc5142adc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.919150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216424-e577-48a5-9a68-925dc5142adc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.929513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:02.930123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:02.930404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:02.930677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:02.931607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:02.931912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:02.933363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:02.943800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96216424-e577-48a5-9a68-925dc5142adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-96216424-e577-48a5-9a68-925dc5142adc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:02.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:02.950469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:02.951306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 269/542] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:02.956683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96216424-e577-48a5-9a68-925dc5142adc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:02.961416 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 274/543] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:02.972075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:03.072383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96216424-e577-48a5-9a68-925dc5142adc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:03.112025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:03.113563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:03.114058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:03.114535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:03.115541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:03.116138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:03.117709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:03.127103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 270/544] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:03.135037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:03.160513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:03.171106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efa4e6-2e92-4d1b-a2ee-7d2f03ac2573 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:03.198940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:03.238139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48156b-c202-4333-a89e-1b79401b6c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:03.239018 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48156b-c202-4333-a89e-1b79401b6c1d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:03.239303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48156b-c202-4333-a89e-1b79401b6c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:03.239573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48156b-c202-4333-a89e-1b79401b6c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:03.240301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48156b-c202-4333-a89e-1b79401b6c1d None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=f7aa31ea778140189b041939e3f01aad) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:03.240607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48156b-c202-4333-a89e-1b79401b6c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:03.242303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48156b-c202-4333-a89e-1b79401b6c1d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:03.255559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 275/545] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:02 2022] GET /identity/v3/role_assignments?user.id=f7aa31ea778140189b041939e3f01aad&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:03.295606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee0c0-45d2-4481-aa58-d4f824624d13 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:03.295955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee0c0-45d2-4481-aa58-d4f824624d13 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:03.296194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee0c0-45d2-4481-aa58-d4f824624d13 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:03.297887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 276/546] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:03.304618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd2431-60ba-4526-bf3e-01ce5e5c8746 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:03.304995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd2431-60ba-4526-bf3e-01ce5e5c8746 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:03.305339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd2431-60ba-4526-bf3e-01ce5e5c8746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:03.311240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d762a-4f59-4dbc-ab41-a2cadfa0e67e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:03.335453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-cafd2431-60ba-4526-bf3e-01ce5e5c8746 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:03.336697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cafd2431-60ba-4526-bf3e-01ce5e5c8746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:03.371914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:03.372638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:03.372922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:03.373194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:03.392922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:03.394407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:03.427660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:03.428592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:03.429538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:03.429538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:03.429538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:03.436797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 271/547] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:03.437802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:03.468945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 277/548] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:03.492039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd2431-60ba-4526-bf3e-01ce5e5c8746 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:03.596984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:03.623369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:03.625300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd2431-60ba-4526-bf3e-01ce5e5c8746 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:03.643636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab980b40-4466-4221-9c16-ab23cdbb585c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:03.681192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:03.682560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:03.682860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:03.683138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:03.683927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:03.684236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:03.685610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:03.702589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 272/549] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:03.704020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:03.732482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a61745-bdc2-4e74-ae7d-232324186058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:03.733136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a61745-bdc2-4e74-ae7d-232324186058 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:03.733412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a61745-bdc2-4e74-ae7d-232324186058 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:03.733680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a61745-bdc2-4e74-ae7d-232324186058 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:03.771949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a61745-bdc2-4e74-ae7d-232324186058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-69a61745-bdc2-4e74-ae7d-232324186058 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:03.777135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:03.778237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:03.779112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:03.787465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 278/550] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:03.793852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a61745-bdc2-4e74-ae7d-232324186058 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:03.812485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fad31-701c-47f0-8992-9dffe1ff6133 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:03.848205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:03.848847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:03.849131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:03.849410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:03.858176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:03.859656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:03.871548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:03.872690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:03.873632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:03.874983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:03.877323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 273/551] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:03.878517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:03.981537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a61745-bdc2-4e74-ae7d-232324186058 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:04.019614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:04.020219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.020498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.020769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.021431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:04.021738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:04.023133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:04.037007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb8e19-4bf0-4475-ab6f-cf01fbe7aa0a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:04.041627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 279/552] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:04.045115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:04.061714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:04.120412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:04.121022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.121300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.121572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.122305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:04.122611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:04.124068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:04.166905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 274/553] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 12:04:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:04.172550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:04.186157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61fdb-fbe4-4fb3-b2a7-4f2ffe1b19a2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:04.251868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:04.252478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.252757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.253027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.274230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:04.276366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:04.279348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:04.312852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:04.313859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:04.314761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:04.314761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:04.314761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:04.314761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 280/554] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:04.321337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:04.354688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbfb8b-b57e-49fc-ac87-7c15c31392f3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:04.389646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:04.406231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4302a-4870-4bb4-939c-a77b32e47259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:04.406848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4302a-4870-4bb4-939c-a77b32e47259 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.407157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4302a-4870-4bb4-939c-a77b32e47259 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.407430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4302a-4870-4bb4-939c-a77b32e47259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.436300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cdcc3-9a4b-4381-ae10-a719a3673fa2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:04.472721 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe28e4-44bf-43c1-914b-928c76542158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:04.473469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe28e4-44bf-43c1-914b-928c76542158 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.473848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe28e4-44bf-43c1-914b-928c76542158 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.474214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe28e4-44bf-43c1-914b-928c76542158 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.474947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe28e4-44bf-43c1-914b-928c76542158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:04.475380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe28e4-44bf-43c1-914b-928c76542158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:04.477034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe28e4-44bf-43c1-914b-928c76542158 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:04.491349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 281/555] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:04.494442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333a00-8234-4a3c-8555-993ee151a9a3 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.494442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333a00-8234-4a3c-8555-993ee151a9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.494442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333a00-8234-4a3c-8555-993ee151a9a3 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.499400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 282/556] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:04.500328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd333a00-8234-4a3c-8555-993ee151a9a3 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:04.501146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4302a-4870-4bb4-939c-a77b32e47259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:04.502669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4302a-4870-4bb4-939c-a77b32e47259 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:04.581694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:04.625745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd333a00-8234-4a3c-8555-993ee151a9a3 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:04.630124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 275/557] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:04:04.631648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67164aa2-4b65-431b-ac37-ae56e0607c0d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.632046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67164aa2-4b65-431b-ac37-ae56e0607c0d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.632457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67164aa2-4b65-431b-ac37-ae56e0607c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.651288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-67164aa2-4b65-431b-ac37-ae56e0607c0d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:04.652466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:04.653118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-67164aa2-4b65-431b-ac37-ae56e0607c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:04.673852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:04.674485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.674810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.675111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.685828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:04.687896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:04.703556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:04.706063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 283/558] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:04:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:04.707860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:04.733196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 276/559] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:04.734421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67164aa2-4b65-431b-ac37-ae56e0607c0d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:04.827156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d069-ae32-4973-8928-a2f66ffa5466 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:04.833534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:04.850219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67164aa2-4b65-431b-ac37-ae56e0607c0d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:04.862231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:04.862860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.863172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.863703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.875243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:04.876725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:04.886772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19f5b-5182-489f-bd81-46702ac86bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:04.887492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19f5b-5182-489f-bd81-46702ac86bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.887877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19f5b-5182-489f-bd81-46702ac86bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.888210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19f5b-5182-489f-bd81-46702ac86bc7 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.888996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e19f5b-5182-489f-bd81-46702ac86bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:04.889303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e19f5b-5182-489f-bd81-46702ac86bc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:04.890712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e19f5b-5182-489f-bd81-46702ac86bc7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:04.891918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:04.892836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:04.894454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:04.894454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:04.894454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:04.894454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 284/560] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:04.894454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:04.914507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 277/561] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:04:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:04.919966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038fcce-a9a2-4968-9859-88f127aa573e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:04.920258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038fcce-a9a2-4968-9859-88f127aa573e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:04.920620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038fcce-a9a2-4968-9859-88f127aa573e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:04.921916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 278/562] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:04.923747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038fcce-a9a2-4968-9859-88f127aa573e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:04.930260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.005512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c106-e97a-447b-aa15-55579b777617 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.042009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038fcce-a9a2-4968-9859-88f127aa573e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.044272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc32b47-15b5-4478-824a-0e3d35330740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.044980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc32b47-15b5-4478-824a-0e3d35330740 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.045337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc32b47-15b5-4478-824a-0e3d35330740 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.045680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc32b47-15b5-4478-824a-0e3d35330740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.057438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc32b47-15b5-4478-824a-0e3d35330740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-5bc32b47-15b5-4478-824a-0e3d35330740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:05.062548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:05.063657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 285/563] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:05.064530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc32b47-15b5-4478-824a-0e3d35330740 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.082516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a39b1a-d9c0-40fc-bf09-334ddde5a9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.083069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a39b1a-d9c0-40fc-bf09-334ddde5a9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.083340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a39b1a-d9c0-40fc-bf09-334ddde5a9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.083605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a39b1a-d9c0-40fc-bf09-334ddde5a9e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.084237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a39b1a-d9c0-40fc-bf09-334ddde5a9e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:05.084533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a39b1a-d9c0-40fc-bf09-334ddde5a9e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.085862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a39b1a-d9c0-40fc-bf09-334ddde5a9e6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.098971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 279/564] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:05.104519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909e31-2e45-4ba5-b709-ea6c3142aa9f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.104806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909e31-2e45-4ba5-b709-ea6c3142aa9f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.105077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909e31-2e45-4ba5-b709-ea6c3142aa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.116234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.128133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-02909e31-2e45-4ba5-b709-ea6c3142aa9f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:05.129314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-02909e31-2e45-4ba5-b709-ea6c3142aa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:05.168537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.177947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc32b47-15b5-4478-824a-0e3d35330740 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.203556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 280/565] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:05.212096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02909e31-2e45-4ba5-b709-ea6c3142aa9f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.214517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.215797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.216279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.216767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.229676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.231527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:05.248110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:05.249141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:05.250085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:05.250085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:05.250085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:05.250085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 286/566] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:05.250085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.332895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02909e31-2e45-4ba5-b709-ea6c3142aa9f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.363142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1a919-fd06-4aa0-80ae-8e4a78df5205 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.371146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.371794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.372073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.372346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.373011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:05.373372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.374780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.376243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.388337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 281/567] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:05.388337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.403830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.404452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.404729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.405012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.411882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.413319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:05.425116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:05.426048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:05.426953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:05.426953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:05.426953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:05.426953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:05.426953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:05.426953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:05.426953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:05.426953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:05.430886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 287/568] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:05.432466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.435222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.491311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134a1da-de2f-4342-aecf-44c0fe738c13 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.525522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9906a-fbec-48f9-9379-deff6b1d9456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.526202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9906a-fbec-48f9-9379-deff6b1d9456 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.526481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9906a-fbec-48f9-9379-deff6b1d9456 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.526756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9906a-fbec-48f9-9379-deff6b1d9456 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.532158 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35cc0f-916e-48e8-9462-c1073453ab82 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.536858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9906a-fbec-48f9-9379-deff6b1d9456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-72d9906a-fbec-48f9-9379-deff6b1d9456 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:05.542728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:05.543854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 282/569] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:05.544728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9906a-fbec-48f9-9379-deff6b1d9456 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.588598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.589140 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.589417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.589683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.590313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:05.590614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.592012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.604201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 288/570] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:05.604804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.612973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.657770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9906a-fbec-48f9-9379-deff6b1d9456 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.664515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.691192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d99af-88d3-4524-bfcd-f2ecb7cb7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.691862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d99af-88d3-4524-bfcd-f2ecb7cb7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.692145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d99af-88d3-4524-bfcd-f2ecb7cb7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.692418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d99af-88d3-4524-bfcd-f2ecb7cb7bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.702397 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f2285-6c4f-4424-aedb-9faad9fb5cf9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.702690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d99af-88d3-4524-bfcd-f2ecb7cb7bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.704183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d99af-88d3-4524-bfcd-f2ecb7cb7bf2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-672d99af-88d3-4524-bfcd-f2ecb7cb7bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:05.718448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:05.719672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:05.720856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:05.720856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:05.720856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:05.720856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 283/571] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:05.720856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b6240-f38f-42be-a54f-d57acb0ef537 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.721196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b6240-f38f-42be-a54f-d57acb0ef537 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.721196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b6240-f38f-42be-a54f-d57acb0ef537 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.722576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 284/572] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:05.723140 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b6240-f38f-42be-a54f-d57acb0ef537 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.741288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.741962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.742237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.742504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.743219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:05.743526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.745587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.757664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 289/573] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:05.758477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.827330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b6240-f38f-42be-a54f-d57acb0ef537 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.836051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.858563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b82f35-c8ef-41e0-84b5-7073d37167fe None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:05.861375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.861975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.862255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.862528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.863293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:05.863848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.865196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.873902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 285/574] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:05.874699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:05.882040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:05.893575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933f1cb-1407-4dfc-bb6b-b3091de1fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:05.894304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933f1cb-1407-4dfc-bb6b-b3091de1fe43 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.894592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933f1cb-1407-4dfc-bb6b-b3091de1fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.894868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933f1cb-1407-4dfc-bb6b-b3091de1fe43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.902086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933f1cb-1407-4dfc-bb6b-b3091de1fe43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:05.903568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933f1cb-1407-4dfc-bb6b-b3091de1fe43 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-b933f1cb-1407-4dfc-bb6b-b3091de1fe43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:05.915157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:05.916727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:05.917848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 290/575] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:05.918447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a89a32-e6a4-4d48-bccb-dbc4673683c8 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:05.918864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a89a32-e6a4-4d48-bccb-dbc4673683c8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:05.918864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a89a32-e6a4-4d48-bccb-dbc4673683c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:05.936578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-06a89a32-e6a4-4d48-bccb-dbc4673683c8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:05.937776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-06a89a32-e6a4-4d48-bccb-dbc4673683c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:05.976334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4640a-9aa3-4429-bba8-e80403083f86 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.002689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 291/576] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:06.003610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a89a32-e6a4-4d48-bccb-dbc4673683c8 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.010898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.011527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.011830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.012097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.012727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:06.013022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.014386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.025787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 286/577] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:06.026665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.057666 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:06.121790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a89a32-e6a4-4d48-bccb-dbc4673683c8 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.128933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8f049-10cc-45ca-bc9e-dcdd0ded4839 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.135377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:06.155455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.156051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.156327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.156596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.157313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=a66a21436030409484b17f803d0e5903) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:06.157614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.159052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.162304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.162917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.163221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.163490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.164490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 292/578] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/role_assignments?user.id=a66a21436030409484b17f803d0e5903&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:06.165272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.182442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.183957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:06.199762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:06.200988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:06.202059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:06.202059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:06.202059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:06.202059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 287/579] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:06.202059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.297485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d8596-cb78-4514-a938-648088f95617 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.298076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:06.316635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31c07b-9a73-41fb-b915-35818c76de7f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.338027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.338787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.339189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.339564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.340367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:06.340786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.342312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.355241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 293/580] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:06.355401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627281d-0046-4830-9a31-7d31f81e18d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.355896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627281d-0046-4830-9a31-7d31f81e18d3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.356240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627281d-0046-4830-9a31-7d31f81e18d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.356518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627281d-0046-4830-9a31-7d31f81e18d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.360684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.368390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627281d-0046-4830-9a31-7d31f81e18d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c627281d-0046-4830-9a31-7d31f81e18d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:06.373701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:06.379538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 288/581] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:06.380613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627281d-0046-4830-9a31-7d31f81e18d3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.406711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:06.473463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a65d4-7755-4a50-8daa-7c7b33ac891f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.482876 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627281d-0046-4830-9a31-7d31f81e18d3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.509918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.510541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.510818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.511114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.518233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.519615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.519740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.520251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.520491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.520761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.521398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:06.521700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.523059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:06.531422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:06.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 294/582] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:06.540048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.544869 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 289/583] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:06.545678 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.548344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:06.647696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254ea1d-3d30-4a15-a6b6-94ccc3ab27b0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.662349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:06.669466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d9a5-d2b8-43ab-8c9e-9bc7d0244747 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.686506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.687475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.687770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.688094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.688766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:06.689069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.690844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.699884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 290/584] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:06.700701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.710925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793d44d-a61e-4d36-acfd-114c869cfb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.711870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793d44d-a61e-4d36-acfd-114c869cfb5a None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.712371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793d44d-a61e-4d36-acfd-114c869cfb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.712854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793d44d-a61e-4d36-acfd-114c869cfb5a None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/cb2075e9fff3459881c13f81456ab198/roles/2e4b771225e74dfc974d2ba9b872c77f` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.730569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793d44d-a61e-4d36-acfd-114c869cfb5a None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=cb2075e9fff3459881c13f81456ab198, role_id=2e4b771225e74dfc974d2ba9b872c77f)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.732041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793d44d-a61e-4d36-acfd-114c869cfb5a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.767606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:06.806549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 295/585] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:06 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/cb2075e9fff3459881c13f81456ab198/roles/2e4b771225e74dfc974d2ba9b872c77f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:06.808085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793d44d-a61e-4d36-acfd-114c869cfb5a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.809578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453a56b-0afe-4317-b3df-47de49df3713 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.854751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.855397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.855676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.855944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:06.856862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:06.857171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:06.858655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:06.885699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 291/586] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:06.886706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:06.951688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793d44d-a61e-4d36-acfd-114c869cfb5a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:06.994715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:06.995432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:06.996514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:06.996514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.009204 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.011361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.022812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091a6d0-ec0e-47c2-8de2-3a87196cedaa None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:07.024191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:07.025864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:07.026856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 296/587] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:07.027643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.035914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:07.056835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:07.075244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.076220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.076673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.077384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.093535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.096764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:07.116823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:07.118029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:07.119509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:07.119509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:07.119509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:07.121208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 292/588] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:07.126738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.164216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ccf2-21e1-4d14-9808-b5356334cb09 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.213917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.214526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.214805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.215098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.226695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.228414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.240092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfac8f-961f-4425-87b1-9f60ff9d592d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:07.247424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:07.248377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:07.249359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:07.249359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:07.249359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:07.249359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 297/589] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:07.250677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.285579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.286347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.286630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.287031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.287811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:07.288116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.289712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.314763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 293/590] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:07.315698 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.324420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:07.353568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:07.385118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d179d1f-dfb5-4aa9-a34a-131525be6010 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.432294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.432607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069839d3-39d8-43ae-afd9-b5f4650446bd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.433326 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.433846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.434533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.436256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:07.436756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.438291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.454990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 298/591] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:07.456220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.473408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.473952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.474224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.474496 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.475165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:07.475467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.476917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.494471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 294/592] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:07.494471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.599322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:07.601087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752cbc4-bf53-4083-ae34-54c76b29f971 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.616639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0029a-16d0-4c23-ac9a-ab25d718d92f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.622931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:07.640931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.641558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.641837 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.642109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.653409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.654860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.669024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.669916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.670345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.670766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:07.671698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:07.672845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:07.674104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:07.674104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:07.674104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:07.674104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 299/593] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:07.674104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.683159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.685407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:07.699970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:07.701125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 12:04:07.702241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:07.705764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 295/594] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:07.706602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.806946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefb9f5-4c1a-410d-b4ca-2e64dc361c2f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.824735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3579da1-05be-4202-bb29-f36a3406768e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:07.843281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.843908 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.844186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.844464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.852593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.854681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:07.867038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:07.868154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:07.869913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:07.869913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:07.869913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:07.869913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:07.869913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:07.869913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:07.869913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:07.869913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:07.872343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:07.873193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:07.873398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 300/595] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:07.873797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:07.874215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:07.874315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.875207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:07.875670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:07.877570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:07.895548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 296/596] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:07.896674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:07.916894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:07.919066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:07.997371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eadbb-9ca1-49e9-8440-88d7e23d2b9d None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.011307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bc82b-4f21-4a1c-a2d4-3b251512d5e8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.036011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.036631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.036956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.037176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.037929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:08.038234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.039847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.052354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.053216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.053644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.054078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.054913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:08.055387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.057356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.064861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 301/597] 10.222.0.30 () {56 vars in 1139 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:08.065766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.070506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 297/598] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:08.071529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.155509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:08.171784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:08.184827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a61b0-fd3f-4be4-83ee-d7a1236a8b34 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.186188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49da9-7919-4ed5-9e77-992470a1f98c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.222636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.223298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.223577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.223840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.224601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.225401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.225893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.227329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.234921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.235388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.236406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.237278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:08.249826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:08.250904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:08.252019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:08.252019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:08.252019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:08.252019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:08.252019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:08.252019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:08.252019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:08.252019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:08.256797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:08.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:08.259955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:08.259955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:08.259955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:08.259955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 298/599] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:08.259955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.261928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 302/600] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:08.262991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.365520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345694c-7113-4785-a1de-e20bd029007a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.368115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2a62a-74f5-4a71-b1e7-f2e5970e4f19 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.390574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:08.401636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac751d-26a6-4e70-ab4e-43660270006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.402295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac751d-26a6-4e70-ab4e-43660270006f None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.402578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac751d-26a6-4e70-ab4e-43660270006f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.402901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac751d-26a6-4e70-ab4e-43660270006f None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/6b5ee9e4388d4bbabc7f1e9e3ca50b67/roles/2e4b771225e74dfc974d2ba9b872c77f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.405949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.406567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.406829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.407134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.407777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:08.408081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.409468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.420267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac751d-26a6-4e70-ab4e-43660270006f None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=6b5ee9e4388d4bbabc7f1e9e3ca50b67, role_id=2e4b771225e74dfc974d2ba9b872c77f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.421694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac751d-26a6-4e70-ab4e-43660270006f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.422511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 303/601] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:08.424155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.437393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:08.473077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 299/602] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:08 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/6b5ee9e4388d4bbabc7f1e9e3ca50b67/roles/2e4b771225e74dfc974d2ba9b872c77f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:08.474544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dbcfb-d0cb-4b11-92d3-20008f606a28 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.474828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dbcfb-d0cb-4b11-92d3-20008f606a28 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.475146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dbcfb-d0cb-4b11-92d3-20008f606a28 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.476746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 300/603] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:08.482121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7dbcfb-d0cb-4b11-92d3-20008f606a28 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.569010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d142b7-c613-4313-a756-20fd036dc718 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.605344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7dbcfb-d0cb-4b11-92d3-20008f606a28 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.611672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1beec-364f-4c99-9a6f-b68973ff4691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.612636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1beec-364f-4c99-9a6f-b68973ff4691 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.613095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1beec-364f-4c99-9a6f-b68973ff4691 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.613825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1beec-364f-4c99-9a6f-b68973ff4691 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.614707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1beec-364f-4c99-9a6f-b68973ff4691 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:08.615274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1beec-364f-4c99-9a6f-b68973ff4691 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.617091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1beec-364f-4c99-9a6f-b68973ff4691 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.632768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 304/604] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:08.634380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4979-1172-4f13-9534-6c4680fc7f2c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.634812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4979-1172-4f13-9534-6c4680fc7f2c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.635248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4979-1172-4f13-9534-6c4680fc7f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.646274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.646909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.647220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.647492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.655418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.656933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.657111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f65a4979-1172-4f13-9534-6c4680fc7f2c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:08.658635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f65a4979-1172-4f13-9534-6c4680fc7f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:08.668977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:08.669934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:08.670842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 301/605] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:08.672287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.709726 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:08.735800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:08.742037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 305/606] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:08.742880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4979-1172-4f13-9534-6c4680fc7f2c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.795113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0470c13-14fd-4467-8dd2-4f95a7eafe62 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.831395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3623b3-34c5-4feb-b9f0-2fc83d32a950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.832055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3623b3-34c5-4feb-b9f0-2fc83d32a950 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.832333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3623b3-34c5-4feb-b9f0-2fc83d32a950 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.832604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3623b3-34c5-4feb-b9f0-2fc83d32a950 None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/e0792940b93b4c1998be971f6143a514/roles/2e4b771225e74dfc974d2ba9b872c77f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.848942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3623b3-34c5-4feb-b9f0-2fc83d32a950 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=e0792940b93b4c1998be971f6143a514, role_id=2e4b771225e74dfc974d2ba9b872c77f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.849279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4979-1172-4f13-9534-6c4680fc7f2c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:08.850373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3623b3-34c5-4feb-b9f0-2fc83d32a950 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.888820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:08.889418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:08.889727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:08.893375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:08.893375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:08.893375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:08.893375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:08.905793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 306/607] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:08.906421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.950631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 302/608] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:08 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/e0792940b93b4c1998be971f6143a514/roles/2e4b771225e74dfc974d2ba9b872c77f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:08.959039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3623b3-34c5-4feb-b9f0-2fc83d32a950 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:08.987503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:09.034938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bd0a0-e9be-44b5-96fd-c59cdc26cf03 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:09.073404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa920-262c-478c-a6e9-8b8c16ada350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:09.074154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa920-262c-478c-a6e9-8b8c16ada350 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.074512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa920-262c-478c-a6e9-8b8c16ada350 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.074899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa920-262c-478c-a6e9-8b8c16ada350 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.090455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaa920-262c-478c-a6e9-8b8c16ada350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-52eaa920-262c-478c-a6e9-8b8c16ada350 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:09.097596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:09.098761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:09.099825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:09.100487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 307/609] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:09.101517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1278ed-d3cd-412d-b100-354a9aca123a None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.101741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1278ed-d3cd-412d-b100-354a9aca123a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.103456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:09.103793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1278ed-d3cd-412d-b100-354a9aca123a None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.106124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 308/610] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:09.106961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1278ed-d3cd-412d-b100-354a9aca123a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:09.137881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3623b3-34c5-4feb-b9f0-2fc83d32a950 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:09.189270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a01141-e4bd-4d7d-ba7a-8b07acfe5f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:09.190811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a01141-e4bd-4d7d-ba7a-8b07acfe5f96 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.191272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a01141-e4bd-4d7d-ba7a-8b07acfe5f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.191698 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a01141-e4bd-4d7d-ba7a-8b07acfe5f96 None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/f7aa31ea778140189b041939e3f01aad/roles/2e4b771225e74dfc974d2ba9b872c77f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.212606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a01141-e4bd-4d7d-ba7a-8b07acfe5f96 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=f7aa31ea778140189b041939e3f01aad, role_id=2e4b771225e74dfc974d2ba9b872c77f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:09.214953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a01141-e4bd-4d7d-ba7a-8b07acfe5f96 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:09.230865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1278ed-d3cd-412d-b100-354a9aca123a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:09.249956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:09.267296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6e3a9-97b6-4333-9a39-27267a4ca58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:09.267899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e3a9-97b6-4333-9a39-27267a4ca58c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.268186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e3a9-97b6-4333-9a39-27267a4ca58c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.268459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e3a9-97b6-4333-9a39-27267a4ca58c None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.269126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e3a9-97b6-4333-9a39-27267a4ca58c None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:09.269426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e3a9-97b6-4333-9a39-27267a4ca58c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:09.270852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e3a9-97b6-4333-9a39-27267a4ca58c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:09.280132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 309/611] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:09.281140 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faf089-0c01-4247-8ef2-ce498d622e09 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.281417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faf089-0c01-4247-8ef2-ce498d622e09 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.281689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faf089-0c01-4247-8ef2-ce498d622e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.293010 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 303/612] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:08 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/f7aa31ea778140189b041939e3f01aad/roles/2e4b771225e74dfc974d2ba9b872c77f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:09.294900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a01141-e4bd-4d7d-ba7a-8b07acfe5f96 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:09.315105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-80faf089-0c01-4247-8ef2-ce498d622e09 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:09.317328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-80faf089-0c01-4247-8ef2-ce498d622e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:09.431308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 310/613] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:09.450312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faf089-0c01-4247-8ef2-ce498d622e09 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:09.534809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a01141-e4bd-4d7d-ba7a-8b07acfe5f96 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:09.579703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faf089-0c01-4247-8ef2-ce498d622e09 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:09.580340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:09.581226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.581714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.582232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.585033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:09.594831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:09.597916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:09.599735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:09.617175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:09.618190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:09.619612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:09.619612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:09.619612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:09.622151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 304/614] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:09.623963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c911a-5c6d-463e-aeb8-d54daa42f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:09.624223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c911a-5c6d-463e-aeb8-d54daa42f0ca None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.626216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c911a-5c6d-463e-aeb8-d54daa42f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.626216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c911a-5c6d-463e-aeb8-d54daa42f0ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.627500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:09.638335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c911a-5c6d-463e-aeb8-d54daa42f0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-b70c911a-5c6d-463e-aeb8-d54daa42f0ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:09.644188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:09.645371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:09.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:09.650505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 311/615] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:09.654734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c911a-5c6d-463e-aeb8-d54daa42f0ca None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:09.769259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c911a-5c6d-463e-aeb8-d54daa42f0ca None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:09.805531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:09.806243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.806728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.807029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.807703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:09.808006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:09.809488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:09.818214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 312/616] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:09.823725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:09.846809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da80247-6b82-459d-8698-817947f8d94f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:09.883389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:09.928242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:09.928853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.929134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.929406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.930052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:09.930355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:09.937023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce44e5-5599-4bc2-99dd-f9dd19da2ce1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:09.944410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:09.973376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 305/617] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:09.974896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:09.975678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:09.976105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:09.976480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:09.979233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:09.990462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:09.992987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:10.009512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:10.010886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:10.011994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:10.011994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:10.011994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:10.011994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 313/618] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:10.016611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:10.087010 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:10.165495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606050-9be2-4118-b2b7-0d2679a9fac5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:10.192767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:10.207433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:10.208632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.209137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.209794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.218175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:10.220013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:10.222183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81cda7-33a8-432f-b3a4-0c051f01945e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:10.234290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:10.235354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:10.236342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:10.239373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 306/619] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:10.245351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:10.287579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:10.288618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.289505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.291765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.292651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:10.293187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:10.294781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:10.325421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 314/620] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:10.330480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:10.387873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539dbbf-0322-4c6f-9e39-99915627193f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:10.396508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:10.427967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:10.429094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.429594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.430072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.431280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:10.432139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:10.433792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:10.453697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 307/621] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 12:04:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:10.458552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:10.546995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe0aa2-e72e-452b-825f-51465b24d357 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:10.601430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40e06-1748-4a4e-b560-53eee3c1fc0e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:10.611710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:10.612696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.612696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.612875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.620895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:10.622340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:10.644573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:10.645192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:10.646150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:10.647171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:10.648564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.649079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.649558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.656096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 315/622] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:04:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:10.661525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:10.662701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:10.664437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:10.670618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:10.683674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:10.688660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:10.690222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:10.690222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:10.690222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:10.690222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 308/623] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:10.691490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:10.718514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:10.806903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f0f1-ec8f-4c6a-8eee-11ceceb23336 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:10.844629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159724e-c723-40f4-9f83-659268ea4d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:10.844881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259b318-ab09-402d-bc65-d521f31cbd9f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:10.845236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159724e-c723-40f4-9f83-659268ea4d02 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.845513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159724e-c723-40f4-9f83-659268ea4d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.845801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159724e-c723-40f4-9f83-659268ea4d02 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.846425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159724e-c723-40f4-9f83-659268ea4d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:10.846725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159724e-c723-40f4-9f83-659268ea4d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:10.848234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159724e-c723-40f4-9f83-659268ea4d02 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:10.861521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 309/624] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:10.862920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7644c-848c-4ec1-ae73-d25c16a6f5ad None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.863230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7644c-848c-4ec1-ae73-d25c16a6f5ad None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.863755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7644c-848c-4ec1-ae73-d25c16a6f5ad None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.865302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 310/625] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:10.871024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81766c3-03a4-43cd-9d57-d8a388577731 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.871366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81766c3-03a4-43cd-9d57-d8a388577731 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.871569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81766c3-03a4-43cd-9d57-d8a388577731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.893012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b81766c3-03a4-43cd-9d57-d8a388577731 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:10.893205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:10.893613 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:10.893898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:10.894166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:10.894944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:10.895235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:10.896289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b81766c3-03a4-43cd-9d57-d8a388577731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:10.896665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:10.918781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 316/626] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:04:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:10.919723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:10.980663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 311/627] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:04:10.981896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81766c3-03a4-43cd-9d57-d8a388577731 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.000997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:11.013654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:11.057766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688f57b-da40-43b0-81de-46227754cf20 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:11.097556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:11.098207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.098707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.098817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.111964 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:11.113449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:11.124752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81766c3-03a4-43cd-9d57-d8a388577731 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:11.132662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:11.133604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:11.135083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:11.135083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:11.135083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:11.135083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:11.135083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:11.135083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:11.135083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:11.135083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:11.139143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 317/628] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:11.140177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.169517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493a300-b0e5-43b0-9722-a5be4e84b7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:11.170048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493a300-b0e5-43b0-9722-a5be4e84b7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.170327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493a300-b0e5-43b0-9722-a5be4e84b7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.170602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493a300-b0e5-43b0-9722-a5be4e84b7a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.241724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493a300-b0e5-43b0-9722-a5be4e84b7a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:11.243200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493a300-b0e5-43b0-9722-a5be4e84b7a2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:11.250381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009016e-d436-4a8d-bb54-ec269d90c6d7 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:11.251944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:11.274104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:11.285747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d305d-5e3a-4f18-a764-d07655f36325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:11.286372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d305d-5e3a-4f18-a764-d07655f36325 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.286651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d305d-5e3a-4f18-a764-d07655f36325 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.286915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d305d-5e3a-4f18-a764-d07655f36325 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.297269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d305d-5e3a-4f18-a764-d07655f36325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-f13d305d-5e3a-4f18-a764-d07655f36325 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:11.301897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:11.302851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:11.303759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:11.308111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 318/629] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:11.308723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d305d-5e3a-4f18-a764-d07655f36325 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.357857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 312/630] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:04:11.359586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a1322-00ec-4ae2-a27e-38395dde51ba None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.359941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a1322-00ec-4ae2-a27e-38395dde51ba None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.360286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a1322-00ec-4ae2-a27e-38395dde51ba None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.361748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 313/631] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:11.366447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a1322-00ec-4ae2-a27e-38395dde51ba None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.425469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d305d-5e3a-4f18-a764-d07655f36325 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:11.464170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:11.464778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.465055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.465328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.466020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:11.466318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:11.467781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:11.478048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a1322-00ec-4ae2-a27e-38395dde51ba None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:11.486037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 319/632] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:11.486037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.486245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:11.514886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14af2dd-ce62-4a03-b35a-f00ae759063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:11.515712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af2dd-ce62-4a03-b35a-f00ae759063d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.515991 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af2dd-ce62-4a03-b35a-f00ae759063d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.516577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af2dd-ce62-4a03-b35a-f00ae759063d None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.517520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14af2dd-ce62-4a03-b35a-f00ae759063d None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:11.517824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14af2dd-ce62-4a03-b35a-f00ae759063d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:11.519949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14af2dd-ce62-4a03-b35a-f00ae759063d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:11.535163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 314/633] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:11.535363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:11.535460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8318e-116a-4db4-9f12-9bc9ba661bb1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.535460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8318e-116a-4db4-9f12-9bc9ba661bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.535460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8318e-116a-4db4-9f12-9bc9ba661bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.557205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c0c8318e-116a-4db4-9f12-9bc9ba661bb1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:11.558723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c0c8318e-116a-4db4-9f12-9bc9ba661bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:11.590594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a896dc-056b-4770-9d50-8dc4c8431963 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:11.626634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:11.627286 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.627614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.627885 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.642921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 315/634] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:11.642921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8318e-116a-4db4-9f12-9bc9ba661bb1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.645574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:11.647049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:11.662829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:11.663875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:11.664779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:11.664779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:11.664779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:11.668823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 320/635] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:11.669656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.751963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:11.756211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8318e-116a-4db4-9f12-9bc9ba661bb1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:11.770487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:11.778269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916e56-7e2e-4781-a79a-2c7db722917c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:11.791856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1fe89-4cb7-4469-9ba7-b0f005ad8d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:11.792509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1fe89-4cb7-4469-9ba7-b0f005ad8d17 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.792786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1fe89-4cb7-4469-9ba7-b0f005ad8d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.793057 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1fe89-4cb7-4469-9ba7-b0f005ad8d17 None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/a66a21436030409484b17f803d0e5903/roles/2e4b771225e74dfc974d2ba9b872c77f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.807901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1fe89-4cb7-4469-9ba7-b0f005ad8d17 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=a66a21436030409484b17f803d0e5903, role_id=2e4b771225e74dfc974d2ba9b872c77f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:11.809418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1fe89-4cb7-4469-9ba7-b0f005ad8d17 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:11.812196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:11.812823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:11.813135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:11.813409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:11.824847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:11.826662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:11.840806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:11.841937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:11.843031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:11.843031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:11.843031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:11.843031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 321/636] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:11.843263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.883747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 316/637] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:11 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/a66a21436030409484b17f803d0e5903/roles/2e4b771225e74dfc974d2ba9b872c77f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:11.884697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1fe89-4cb7-4469-9ba7-b0f005ad8d17 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:11.959491 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13632f-0a25-410a-9849-52ee342ad71a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.000737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1fe89-4cb7-4469-9ba7-b0f005ad8d17 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.005798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165d977-832d-45dc-96b6-a27bbd11322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.006372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165d977-832d-45dc-96b6-a27bbd11322e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.006647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165d977-832d-45dc-96b6-a27bbd11322e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.006916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165d977-832d-45dc-96b6-a27bbd11322e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.019740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165d977-832d-45dc-96b6-a27bbd11322e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-a165d977-832d-45dc-96b6-a27bbd11322e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:12.025044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:12.026046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:12.026995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 322/638] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:12.027737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afc9d9-018b-4574-a6b8-6239437fc52f None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.027737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afc9d9-018b-4574-a6b8-6239437fc52f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.028767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afc9d9-018b-4574-a6b8-6239437fc52f None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.029181 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 323/639] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:12.029865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afc9d9-018b-4574-a6b8-6239437fc52f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.046521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:12.050520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.051162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.051444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.051737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.052357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:12.052656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.054006 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.066406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 317/640] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:12.067427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.080084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:12.153990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afc9d9-018b-4574-a6b8-6239437fc52f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.186210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d6c68-b1c9-4b12-b12f-2005924782dd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.189153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca908cd0-e4d5-4d64-87c5-cff757337543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.189153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908cd0-e4d5-4d64-87c5-cff757337543 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.189411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908cd0-e4d5-4d64-87c5-cff757337543 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.189603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908cd0-e4d5-4d64-87c5-cff757337543 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.190248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca908cd0-e4d5-4d64-87c5-cff757337543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:12.190595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca908cd0-e4d5-4d64-87c5-cff757337543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.192028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca908cd0-e4d5-4d64-87c5-cff757337543 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.205116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 324/641] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:12.206624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0196b-4bbe-453e-bb55-b6086d06632a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.207125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0196b-4bbe-453e-bb55-b6086d06632a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.207571 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0196b-4bbe-453e-bb55-b6086d06632a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.225088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-9cd0196b-4bbe-453e-bb55-b6086d06632a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:12.226333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9cd0196b-4bbe-453e-bb55-b6086d06632a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:12.235663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.236902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.237385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.238148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.239586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:12.240095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.242181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.253736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 318/642] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:12.255328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.289307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:12.303875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 325/643] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:12.303875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0196b-4bbe-453e-bb55-b6086d06632a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.342669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:12.377066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84797d08-221d-4658-a53d-eeab0c624fc5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.406121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0196b-4bbe-453e-bb55-b6086d06632a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.414485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.416007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.416289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.416604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.423911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.425485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:12.437694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:12.438681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 319/644] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:12.439658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.444280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.444924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.445205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.445476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.452987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.454429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:12.466784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:12.467762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 326/645] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:12.468874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.532092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:12.557039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a77ce-6ab4-4b57-a34b-72d553493bca None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.581108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5778b-ef4a-4a84-8fdc-ffa448931dbc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.587055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:12.593364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.596239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.596239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.596239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.606005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.607569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.620224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10c67-add3-4950-aa8b-f98a6ccde8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.620904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10c67-add3-4950-aa8b-f98a6ccde8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.621138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10c67-add3-4950-aa8b-f98a6ccde8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.621410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10c67-add3-4950-aa8b-f98a6ccde8a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:12.624160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:12.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:12.626048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:12.626048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:12.626048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:12.626048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 320/646] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:12.626048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.640762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad10c67-add3-4950-aa8b-f98a6ccde8a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-1ad10c67-add3-4950-aa8b-f98a6ccde8a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:12.646307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:12.647410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:12.648500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:12.651064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 327/647] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:12.655145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10c67-add3-4950-aa8b-f98a6ccde8a9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.737072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505973-b7f3-4b3f-b4dc-de688a17b174 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.760069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10c67-add3-4950-aa8b-f98a6ccde8a9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.773662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.774259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.774536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.775392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.776440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:12.776752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.777315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:12.778288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.795096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 321/648] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:12.795096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.802986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.802986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.802986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.802986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.803306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:12.803306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.803755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.827385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 328/649] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:12.828183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.839046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:12.897762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f5444-17ae-4554-abfc-b2deec7d6cf4 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.929339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ce27e-680f-4c3a-b78f-1eda25732214 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:12.933765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.934368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.934645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.934917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.935586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:12.935890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.937245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.954854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 322/650] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:12.954854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:12.964743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:12.965349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:12.965625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:12.965892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:12.966549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:12.966849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:12.968239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:12.976375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 329/651] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:12.977228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:13.018978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:13.072461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f573e6-26fe-4da8-bbd3-213037aeac8f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:13.074377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:13.087998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768cb0e-8b2e-4db1-b82c-be8d138145c6 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:13.117989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:13.117989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.117989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.117989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.118355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=cb2075e9fff3459881c13f81456ab198) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:13.118355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:13.120365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:13.126250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 323/652] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/role_assignments?user.id=cb2075e9fff3459881c13f81456ab198&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:13.135269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:13.147918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea156155-38e2-4031-b215-0a5548ebee6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:13.148627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea156155-38e2-4031-b215-0a5548ebee6e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.149013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea156155-38e2-4031-b215-0a5548ebee6e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.149352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea156155-38e2-4031-b215-0a5548ebee6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.240965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea156155-38e2-4031-b215-0a5548ebee6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:13.242538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea156155-38e2-4031-b215-0a5548ebee6e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:13.251258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88fc8e-db3e-4e66-be5b-0f7719f2762c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:13.286797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:13.298050 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:13.298803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.299997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.299997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.311810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:13.313379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:13.326098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:13.327169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 12:04:13.328402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:13.328967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 324/653] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:13.330224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:13.370900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 330/654] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:04:13.371309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea156155-38e2-4031-b215-0a5548ebee6e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:13.374985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:13.459995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143abe40-b5f4-41ee-a3af-892244f7cd09 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:13.493895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea156155-38e2-4031-b215-0a5548ebee6e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:13.512266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:13.513580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.514041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.514517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.529998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:13.530961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:13.531681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.532077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.532420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.532618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:13.533269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:13.533644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:13.535238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:13.538776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:13.553232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:13.554239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:13.555187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:13.555187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:13.555187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:13.555372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 331/655] 10.222.0.30 () {56 vars in 1137 bytes} [Sun Aug 7 12:04:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:13.557393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 325/656] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:13.562183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:13.563288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:13.659622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:13.677609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38ae1f-b9aa-4297-967e-a9b5815cd678 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:13.690379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ada2c-e509-4499-ac81-4dba14e1b6b0 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:13.714098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3eb4a-fcc5-4c58-8b40-0f9db9a6ce9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:13.714864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3eb4a-fcc5-4c58-8b40-0f9db9a6ce9d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.715266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3eb4a-fcc5-4c58-8b40-0f9db9a6ce9d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.715644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3eb4a-fcc5-4c58-8b40-0f9db9a6ce9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.744426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:13.745360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.745655 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.745932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.760672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:13.762123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:13.782511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:13.783728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:13.784904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:13.784904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:13.784904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:13.787148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 332/657] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:13.788023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:13.800319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3eb4a-fcc5-4c58-8b40-0f9db9a6ce9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:13.801889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3eb4a-fcc5-4c58-8b40-0f9db9a6ce9d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:13.840394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:13.904983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24afbf-ae24-4a50-9b64-d1a3d269f047 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:13.908125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:13.919597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 326/658] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:04:13.920615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3eb4a-fcc5-4c58-8b40-0f9db9a6ce9d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:13.948221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44566c54-68e0-4b1b-8869-e9f5b004cc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:13.949173 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44566c54-68e0-4b1b-8869-e9f5b004cc8f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.949596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44566c54-68e0-4b1b-8869-e9f5b004cc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.950010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44566c54-68e0-4b1b-8869-e9f5b004cc8f None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.950925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44566c54-68e0-4b1b-8869-e9f5b004cc8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:13.951398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44566c54-68e0-4b1b-8869-e9f5b004cc8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:13.952966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44566c54-68e0-4b1b-8869-e9f5b004cc8f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:13.966951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 333/659] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:13.968329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc761-3900-4821-bde9-d7216cfcc70d None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:13.968779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc761-3900-4821-bde9-d7216cfcc70d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:13.969196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc761-3900-4821-bde9-d7216cfcc70d None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:13.974576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 334/660] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:13.975409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc761-3900-4821-bde9-d7216cfcc70d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.033600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3eb4a-fcc5-4c58-8b40-0f9db9a6ce9d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.069073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.069689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.069942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.070206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.070940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:14.071264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.073791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.078866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc761-3900-4821-bde9-d7216cfcc70d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.089134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 327/661] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:14.089134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.089395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:14.125954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cede8-b863-4ed3-a722-3c7cdc78891f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.126584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cede8-b863-4ed3-a722-3c7cdc78891f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.126866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cede8-b863-4ed3-a722-3c7cdc78891f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.127176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cede8-b863-4ed3-a722-3c7cdc78891f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.135177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:14.140196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cede8-b863-4ed3-a722-3c7cdc78891f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.141812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cede8-b863-4ed3-a722-3c7cdc78891f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-162cede8-b863-4ed3-a722-3c7cdc78891f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:14.157182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:14.158179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:14.159126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:14.159126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:14.159126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:14.159126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 335/662] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:14.159777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862683d-6321-4e45-899d-021188435f2d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.159777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862683d-6321-4e45-899d-021188435f2d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.160063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862683d-6321-4e45-899d-021188435f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.178463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2862683d-6321-4e45-899d-021188435f2d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:14.179575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2862683d-6321-4e45-899d-021188435f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:14.203423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46f0ee-d52d-439d-a656-0752015d08f7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.237655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.238274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.238550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.239112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.247107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.248624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.259770 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 336/663] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:14.269366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:14.270511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 12:04:14.271768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:14.272431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862683d-6321-4e45-899d-021188435f2d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.272523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 328/664] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:14.272523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.358184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:14.371205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:14.371371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862683d-6321-4e45-899d-021188435f2d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.375569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec31e-021c-4574-bf31-ef83d8a39ae8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.406438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.407124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.407404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.407681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.415794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.417256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.418282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.419318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.420123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.420582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.421402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:14.421884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.423705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:14.428736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:14.429647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:14.430565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 337/665] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:14.431236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.441525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 329/666] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:14.442360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.538622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa9ee6-7cca-41af-8ded-a57bf9726126 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.563407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd0e6-f748-4848-8313-b8a058a33514 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.575514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.576111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.576386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.576652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.577370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:14.577670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.579112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.597314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 338/667] 10.222.0.30 () {56 vars in 1139 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:14.598107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.605852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1efdd2-28b5-49ef-9303-8be3a17f87f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.606489 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1efdd2-28b5-49ef-9303-8be3a17f87f9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.606764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1efdd2-28b5-49ef-9303-8be3a17f87f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.607060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1efdd2-28b5-49ef-9303-8be3a17f87f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.607894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:14.617915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1efdd2-28b5-49ef-9303-8be3a17f87f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1a1efdd2-28b5-49ef-9303-8be3a17f87f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:14.623508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:14.624460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:14.625398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 330/668] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:14.626116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1efdd2-28b5-49ef-9303-8be3a17f87f9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.645317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:14.706931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5e2fb-0035-44d3-a2f3-23b2df3cc625 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.734577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1efdd2-28b5-49ef-9303-8be3a17f87f9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.743033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.743799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.744073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.744362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.745043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:14.745350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.746776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.758785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 339/669] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:14.759498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.776931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.777553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.777828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.778096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.784911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.786330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:14.797766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:14.803463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 331/670] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:14.804612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.841546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:14.876967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17103ccf-b328-417e-9051-6ae62469900e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.883572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:14.905448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1132c-0d99-43d0-a9c7-951c8d6aef4b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:14.914008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41571-4b4c-4451-83bb-e3e861514c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.914610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41571-4b4c-4451-83bb-e3e861514c88 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.915710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41571-4b4c-4451-83bb-e3e861514c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.915710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41571-4b4c-4451-83bb-e3e861514c88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.943955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:14.944570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:14.945069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:14.945179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:14.945779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:14.946092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.948326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:14.957957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 332/671] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:14.958625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:14.994333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41571-4b4c-4451-83bb-e3e861514c88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:14.995836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41571-4b4c-4451-83bb-e3e861514c88 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.063646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502e398-c7d7-4fd9-9c1a-d29426f9f0ef None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.068628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:15.103704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:15.104658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.105070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.105470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.106340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=6b5ee9e4388d4bbabc7f1e9e3ca50b67) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:15.106779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:15.108388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.110663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 340/672] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:04:15.111798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41571-4b4c-4451-83bb-e3e861514c88 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.119220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 333/673] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:14 2022] GET /identity/v3/role_assignments?user.id=6b5ee9e4388d4bbabc7f1e9e3ca50b67&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:15.120221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.127788 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:15.240221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cc7f5-4765-430d-99b0-4f1bde5b2c4b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.273241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41571-4b4c-4451-83bb-e3e861514c88 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.281796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:15.282786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.283900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.284396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.301297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:15.303209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:15.318882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:15.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:15.320900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:15.320900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:15.320900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:15.327430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 334/674] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:15.328339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.329971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:15.330601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.330878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.331201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.331909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:15.332216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:15.334915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.349112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 341/675] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:15.349956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.381444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:15.400171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:15.450033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9680dd-de9a-4c1f-8001-b280e182ade8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.464328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eaff88-da11-4d98-81c4-b344d2ec7d9b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.496061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:15.496061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.496061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.496061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.501782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7aa86-510b-49df-9eb3-95a78b06c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:15.502419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7aa86-510b-49df-9eb3-95a78b06c812 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.502701 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7aa86-510b-49df-9eb3-95a78b06c812 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.502982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7aa86-510b-49df-9eb3-95a78b06c812 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.503659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7aa86-510b-49df-9eb3-95a78b06c812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:15.504117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7aa86-510b-49df-9eb3-95a78b06c812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:15.504586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:15.505500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7aa86-510b-49df-9eb3-95a78b06c812 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.506021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.518520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 342/676] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:15.520224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f7e3-f28f-4ba5-bd80-aa76c99dc131 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.520729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f7e3-f28f-4ba5-bd80-aa76c99dc131 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.521208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f7e3-f28f-4ba5-bd80-aa76c99dc131 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:15.521692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:15.522654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:15.523953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:15.523953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:15.523953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:15.524399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 343/677] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:15.525663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745f7e3-f28f-4ba5-bd80-aa76c99dc131 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.530084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 335/678] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:15.530915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.620028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:15.636601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745f7e3-f28f-4ba5-bd80-aa76c99dc131 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.644759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb844b-7cce-415a-8ee2-2afad88e1331 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.657884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:15.676489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7237fc9-0e5b-4bc2-a9f6-df992e436428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:15.677169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7237fc9-0e5b-4bc2-a9f6-df992e436428 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.677451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7237fc9-0e5b-4bc2-a9f6-df992e436428 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.677717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7237fc9-0e5b-4bc2-a9f6-df992e436428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.678438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7237fc9-0e5b-4bc2-a9f6-df992e436428 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=e0792940b93b4c1998be971f6143a514) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:15.678915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7237fc9-0e5b-4bc2-a9f6-df992e436428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:15.680263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7237fc9-0e5b-4bc2-a9f6-df992e436428 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.683523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:15.684196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.684855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.685621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.686856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 344/679] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/role_assignments?user.id=e0792940b93b4c1998be971f6143a514&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:15.692392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba484-67a9-424f-b5e8-ad9002dd640f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.692757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba484-67a9-424f-b5e8-ad9002dd640f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.693103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba484-67a9-424f-b5e8-ad9002dd640f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.695923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:15.698315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:15.711727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:15.713350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:15.714498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:15.716292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3eeba484-67a9-424f-b5e8-ad9002dd640f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:15.717904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 336/680] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:15.718559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3eeba484-67a9-424f-b5e8-ad9002dd640f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:15.719181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.807160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 345/681] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:15.808451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeba484-67a9-424f-b5e8-ad9002dd640f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.866419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a29bc-4353-475e-ad8f-968ffe9b21cf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.908897 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:15.909501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:15.909778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:15.910046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:15.910688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:15.910986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:15.912911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:15.925100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:15.935090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 337/682] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:15.936273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:15.950336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeba484-67a9-424f-b5e8-ad9002dd640f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:15.975614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:16.003826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.005153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.005710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.006309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.007873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:16.008386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:16.011200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:16.038144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 346/683] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:16.039328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.061206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d275e-eeef-4791-8ecd-51cd5769f7bd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:16.098025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61e63-a86e-4fa7-90f4-59cd962db7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.098877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e63-a86e-4fa7-90f4-59cd962db7cb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.099575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e63-a86e-4fa7-90f4-59cd962db7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.099575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e63-a86e-4fa7-90f4-59cd962db7cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.110684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61e63-a86e-4fa7-90f4-59cd962db7cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b1a61e63-a86e-4fa7-90f4-59cd962db7cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:16.115961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:16.117096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:16.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:16.120727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 338/684] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:16.121747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61e63-a86e-4fa7-90f4-59cd962db7cb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.174142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc919dd0-1f7e-44c9-a553-c18bae4fe11d None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:16.215832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:16.220682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.221772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.222202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.225660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.229570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61e63-a86e-4fa7-90f4-59cd962db7cb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:16.232767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:16.234244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:16.236060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:16.249821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:16.255086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:16.256014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:16.256014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:16.256014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:16.256014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:16.256014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:16.256014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:16.256014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:16.256014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:16.256902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 347/685] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:16.257603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.271978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355b5e0-71c2-45dc-9953-d0de3108ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.272949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355b5e0-71c2-45dc-9953-d0de3108ae80 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.273431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355b5e0-71c2-45dc-9953-d0de3108ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.273900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355b5e0-71c2-45dc-9953-d0de3108ae80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.368368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355b5e0-71c2-45dc-9953-d0de3108ae80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:16.370883 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355b5e0-71c2-45dc-9953-d0de3108ae80 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:16.375806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb9a26-1ce1-4ec9-a40e-df1e874347b4 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:16.420921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.421538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.421822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.422651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.423425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:16.423734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:16.426157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:16.438612 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 348/686] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:16.439553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.479473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:16.497446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 339/687] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:04:16.498491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355b5e0-71c2-45dc-9953-d0de3108ae80 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.522145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:16.565929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c01453-9745-4fa4-8013-61cd7fc34180 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:16.614883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.615911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.616207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.616483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.617485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:16.617794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:16.620471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:16.622501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355b5e0-71c2-45dc-9953-d0de3108ae80 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:16.639076 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 349/688] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:16.640245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.674065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.674708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.674976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.675296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.688887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:16.690616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:16.711456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:16.712772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:16.713713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:16.713713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:16.713713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:16.713713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 340/689] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:16.713948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.761382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3b235-a214-45f1-a922-16626947fe69 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:16.764078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:16.790044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:16.807454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.808086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.808432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.808705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.820103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:16.821928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:16.832283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac2be-ca2a-4914-8e7b-90571686837d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:16.838753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:16.839766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:16.840731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:16.840731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:16.840731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:16.844548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 350/690] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:16.845431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.880848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:16.881860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:16.882340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:16.882806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:16.883809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=f7aa31ea778140189b041939e3f01aad) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:16.884645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:16.886275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:16.899199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 341/691] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/role_assignments?user.id=f7aa31ea778140189b041939e3f01aad&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:16.901031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:16.982814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ccfb-2fef-49af-91b2-f655909358de None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:17.027029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:17.027029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.027029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.027029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.027483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:17.027790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:17.029889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:17.046912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 351/692] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:17.056281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:17.128885 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:17.148257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae04ef-8ab9-4dd1-a3ba-b625f9377542 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:17.151564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:17.189232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:17.190082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.190506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.190949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.191758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:17.192203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:17.193707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:17.210375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 342/693] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:17.216335 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:17.227439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432cbf-0de5-481f-8bc6-232b06885723 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:17.266501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ba3e1-e19b-461a-b506-56e979e27096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:17.267402 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ba3e1-e19b-461a-b506-56e979e27096 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.267630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ba3e1-e19b-461a-b506-56e979e27096 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.267906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ba3e1-e19b-461a-b506-56e979e27096 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.276678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ba3e1-e19b-461a-b506-56e979e27096 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:17.278250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ba3e1-e19b-461a-b506-56e979e27096 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-973ba3e1-e19b-461a-b506-56e979e27096 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:17.292619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:17.293637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:17.294610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 352/694] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:04:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:17.295244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65adb80-3f80-4e66-80a0-6575a0fb4c84 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.295864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65adb80-3f80-4e66-80a0-6575a0fb4c84 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.296151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65adb80-3f80-4e66-80a0-6575a0fb4c84 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.297619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 353/695] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:17.301760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65adb80-3f80-4e66-80a0-6575a0fb4c84 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:17.412807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:17.423341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65adb80-3f80-4e66-80a0-6575a0fb4c84 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:17.462739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c3740-85e1-447f-b26d-d433c03037fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:17.463570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c3740-85e1-447f-b26d-d433c03037fe None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.463945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c3740-85e1-447f-b26d-d433c03037fe None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.464312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c3740-85e1-447f-b26d-d433c03037fe None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.465063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff045750-64e4-44e6-ad0f-035a56d7e49f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:17.465184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c3740-85e1-447f-b26d-d433c03037fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:17.465596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c3740-85e1-447f-b26d-d433c03037fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:17.467266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c3740-85e1-447f-b26d-d433c03037fe None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:17.486349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 354/696] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:04:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:17.487996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ce685-77bd-4895-9b2a-10d02b18ee19 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.488278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ce685-77bd-4895-9b2a-10d02b18ee19 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.488897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ce685-77bd-4895-9b2a-10d02b18ee19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.503715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:17.504654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.505076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.505693 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.510352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-244ce685-77bd-4895-9b2a-10d02b18ee19 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:17.511757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-244ce685-77bd-4895-9b2a-10d02b18ee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:17.514333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:17.516164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:17.528785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:17.529739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:17.530638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:17.530638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:17.530638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:17.530638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:17.530638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:17.530638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:17.530638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:17.530638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:17.532828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 343/697] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:17.533831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:17.562435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:17.591108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 355/698] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:17.592335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ce685-77bd-4895-9b2a-10d02b18ee19 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:17.662931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e7899-2dea-4ab3-88ae-b877a77f7b14 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:17.685825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:17.704218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b665a3-493b-4354-a28b-75ee599960a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:17.705686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b665a3-493b-4354-a28b-75ee599960a2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.706460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b665a3-493b-4354-a28b-75ee599960a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.707190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b665a3-493b-4354-a28b-75ee599960a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.708843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ce685-77bd-4895-9b2a-10d02b18ee19 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:17.745890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:17.746491 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.746782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.747082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.747751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:17.748056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:17.749537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:17.762464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 356/699] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:17.763442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:17.795640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b665a3-493b-4354-a28b-75ee599960a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:17.797175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b665a3-493b-4354-a28b-75ee599960a2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:17.824445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:17.888370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa2f3b-40cd-4830-9fed-a6313ef50143 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:17.908066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 344/700] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 12:04:17.908066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b665a3-493b-4354-a28b-75ee599960a2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:17.931041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e4f7e-7cb0-4cb5-855d-46de144953f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:17.931739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4f7e-7cb0-4cb5-855d-46de144953f1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.932011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4f7e-7cb0-4cb5-855d-46de144953f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.932282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4f7e-7cb0-4cb5-855d-46de144953f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.941925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:17.944711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e4f7e-7cb0-4cb5-855d-46de144953f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:17.946263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e4f7e-7cb0-4cb5-855d-46de144953f1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-579e4f7e-7cb0-4cb5-855d-46de144953f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:17.961564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:17.963309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 357/701] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b76ce-100e-4877-bba2-6c9d115676d0 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b76ce-100e-4877-bba2-6c9d115676d0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b76ce-100e-4877-bba2-6c9d115676d0 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 358/702] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:17.971380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b76ce-100e-4877-bba2-6c9d115676d0 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.015088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b665a3-493b-4354-a28b-75ee599960a2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:18.038531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:18.049248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:18.049849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.050125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.050391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.051130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=a66a21436030409484b17f803d0e5903) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:18.051433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:18.052859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:18.058644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 345/703] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:17 2022] GET /identity/v3/role_assignments?user.id=a66a21436030409484b17f803d0e5903&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:18.059270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.087490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b76ce-100e-4877-bba2-6c9d115676d0 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:18.138855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed430155-234a-4f8a-aa03-77019bd91b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:18.139867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430155-234a-4f8a-aa03-77019bd91b5c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.140348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430155-234a-4f8a-aa03-77019bd91b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.140804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430155-234a-4f8a-aa03-77019bd91b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.153692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed430155-234a-4f8a-aa03-77019bd91b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:18.155500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed430155-234a-4f8a-aa03-77019bd91b5c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-ed430155-234a-4f8a-aa03-77019bd91b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:18.172468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:18.173506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:18.174424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:18.174424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:18.174424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:18.176677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 359/704] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:18.178410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc9bf8-2dc7-48cf-9b22-781c3b7fac1c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.178880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc9bf8-2dc7-48cf-9b22-781c3b7fac1c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.179348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc9bf8-2dc7-48cf-9b22-781c3b7fac1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.198803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4b7d8-91c2-465d-b8ee-d71c676468c2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:18.201053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f0fc9bf8-2dc7-48cf-9b22-781c3b7fac1c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:18.202257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f0fc9bf8-2dc7-48cf-9b22-781c3b7fac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:18.240895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:18.241502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.241777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.242041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.242678 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:18.242976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:18.244878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:18.251124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:18.261760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 346/705] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:18.262607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.295225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 360/706] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:18.296449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc9bf8-2dc7-48cf-9b22-781c3b7fac1c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.325748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:18.389241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359abdaa-6528-4b52-bf6f-22b5b425c145 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:18.422167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc9bf8-2dc7-48cf-9b22-781c3b7fac1c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:18.432239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:18.432853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.433129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.433399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.434052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:18.434352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:18.435773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:18.448988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 347/707] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:18.449804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.584364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979213-29f0-43c0-8271-c37e91364629 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:18.585153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85910885-f78c-4239-ba2d-32d65381dc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:18.585824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910885-f78c-4239-ba2d-32d65381dc83 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.586169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910885-f78c-4239-ba2d-32d65381dc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.586505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910885-f78c-4239-ba2d-32d65381dc83 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.587273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85910885-f78c-4239-ba2d-32d65381dc83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:18.588774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85910885-f78c-4239-ba2d-32d65381dc83 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:18.594107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:18.622014 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:18.622639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.622915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.623223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.624717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:18.626203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:18.637929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:18.639061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:18.639976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:18.643091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 348/708] 10.222.0.30 () {56 vars in 1129 bytes} [Sun Aug 7 12:04:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:18.644458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.667250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 361/709] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:04:18 2022] POST /identity/v3/users => generated 254 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:18.667250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85910885-f78c-4239-ba2d-32d65381dc83 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.736890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:18.814772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff8b48-339e-4ec9-94d5-767621f37501 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:18.853411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:18.855426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.855918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.856412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.864813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:18.866503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:18.875644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85910885-f78c-4239-ba2d-32d65381dc83 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:18.880254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:18.881186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:18.882131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:18.882131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:18.882131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:18.882131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:18.882131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:18.882131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:18.882131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:18.882131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:18.885968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 349/710] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:18.892514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.914270 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:18.914884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:18.915225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:18.915499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:18.916142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:18.916438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:18.917822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:18.928620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 362/711] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 12:04:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:18.928938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:18.959385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:19.029326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be3720-1762-415a-a2d1-2b0d4c36979b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:19.034606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:19.042585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e576e-ed21-42ef-9f2c-2751ca981b04 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:19.072926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:19.073517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:19.074120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:19.074120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:19.074761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:19.075102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:19.076527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:19.093692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 350/712] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:19.100997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:19.111092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f20114-6d0c-4315-a84a-be538e631578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:19.111879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f20114-6d0c-4315-a84a-be538e631578 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:19.112331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f20114-6d0c-4315-a84a-be538e631578 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:19.112742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f20114-6d0c-4315-a84a-be538e631578 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:19.113548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f20114-6d0c-4315-a84a-be538e631578 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:19.114367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f20114-6d0c-4315-a84a-be538e631578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:19.115898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f20114-6d0c-4315-a84a-be538e631578 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:19.127670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 363/713] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 12:04:18 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:19.223618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:19.233538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835376f-728b-40f8-8672-29cd72c44586 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:19.272382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f839bd-80af-4353-bfc6-d199028f8194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:19.273360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f839bd-80af-4353-bfc6-d199028f8194 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:19.273846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f839bd-80af-4353-bfc6-d199028f8194 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:19.274348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f839bd-80af-4353-bfc6-d199028f8194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:19.275891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f839bd-80af-4353-bfc6-d199028f8194 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=cb2075e9fff3459881c13f81456ab198) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:19.276408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f839bd-80af-4353-bfc6-d199028f8194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:19.278012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f839bd-80af-4353-bfc6-d199028f8194 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:19.285148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 351/714] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:19 2022] GET /identity/v3/role_assignments?user.id=cb2075e9fff3459881c13f81456ab198&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=f72d3388f13149b483355331380f040b => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:19.366717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc9d4-505f-4584-a7c0-f42edd815ba8 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:19.367092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc9d4-505f-4584-a7c0-f42edd815ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:19.367290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc9d4-505f-4584-a7c0-f42edd815ba8 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:19.369505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 364/715] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:19.375453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6f6df-4c3b-4404-9d4f-bea098389a9e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:19.375796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6f6df-4c3b-4404-9d4f-bea098389a9e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:19.376095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6f6df-4c3b-4404-9d4f-bea098389a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:19.399188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-38f6f6df-4c3b-4404-9d4f-bea098389a9e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:19.400992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-38f6f6df-4c3b-4404-9d4f-bea098389a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:19.480890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 352/716] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:19.502813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cc9d4-505f-4584-a7c0-f42edd815ba8 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:19.632921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cc9d4-505f-4584-a7c0-f42edd815ba8 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:19.669458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:19.670085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:19.670364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:19.670634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] PATH_INFO: `/v3/services/network` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:19.671302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:19.672689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:19.684178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:19.685450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:19.686610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 365/717] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:19 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:19.693385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6f6df-4c3b-4404-9d4f-bea098389a9e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:19.813568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:19.829430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6f6df-4c3b-4404-9d4f-bea098389a9e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:19.872720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f3a2d-d1d2-4c71-86c6-6b3714cfd434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:19.873586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3a2d-d1d2-4c71-86c6-6b3714cfd434 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:19.874023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3a2d-d1d2-4c71-86c6-6b3714cfd434 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:19.874444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3a2d-d1d2-4c71-86c6-6b3714cfd434 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:19.875281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f3a2d-d1d2-4c71-86c6-6b3714cfd434 None admin] RBAC: Adding query filter params (name=network) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:19.875729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f3a2d-d1d2-4c71-86c6-6b3714cfd434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:19.877224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f3a2d-d1d2-4c71-86c6-6b3714cfd434 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:19.882338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 353/718] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:19.887834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:19.926593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:20.017385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc1d7a-be99-4b14-a0d7-682bf1172999 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:20.053101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae6f67-45c9-4ebe-b459-cbe5944bf5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:20.053755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae6f67-45c9-4ebe-b459-cbe5944bf5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:20.054033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae6f67-45c9-4ebe-b459-cbe5944bf5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:20.054306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae6f67-45c9-4ebe-b459-cbe5944bf5a5 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:20.054991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae6f67-45c9-4ebe-b459-cbe5944bf5a5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:20.055385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae6f67-45c9-4ebe-b459-cbe5944bf5a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:20.057405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae6f67-45c9-4ebe-b459-cbe5944bf5a5 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:20.062391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 366/719] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:20.393081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1dbb2-74bd-4e58-be70-7f74eff07414 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:20.393755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1dbb2-74bd-4e58-be70-7f74eff07414 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:20.394245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1dbb2-74bd-4e58-be70-7f74eff07414 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:20.396064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 354/720] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:20.401897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b89990-f9a4-4608-8c6d-5c174d029e00 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:20.402190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b89990-f9a4-4608-8c6d-5c174d029e00 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:20.402471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b89990-f9a4-4608-8c6d-5c174d029e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:20.424707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-03b89990-f9a4-4608-8c6d-5c174d029e00 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:20.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-03b89990-f9a4-4608-8c6d-5c174d029e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:20.500880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 367/721] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:20.526721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1dbb2-74bd-4e58-be70-7f74eff07414 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:20.689082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1dbb2-74bd-4e58-be70-7f74eff07414 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:20.733785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519903b-6a6a-45cd-8ceb-d51c9b0704c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:20.734420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519903b-6a6a-45cd-8ceb-d51c9b0704c4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:20.734705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519903b-6a6a-45cd-8ceb-d51c9b0704c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:20.734984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519903b-6a6a-45cd-8ceb-d51c9b0704c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:20.755956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519903b-6a6a-45cd-8ceb-d51c9b0704c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5519903b-6a6a-45cd-8ceb-d51c9b0704c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:20.760913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:20.762075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:20.763123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 355/722] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:20.768395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b89990-f9a4-4608-8c6d-5c174d029e00 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:20.802978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:20.936550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b89990-f9a4-4608-8c6d-5c174d029e00 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:20.982077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:20.982612 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:20.982893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:20.983188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:20.983853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:20.984155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:20.985534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:20.998678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 368/723] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:21.004392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519903b-6a6a-45cd-8ceb-d51c9b0704c4 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:21.030760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:21.141068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519903b-6a6a-45cd-8ceb-d51c9b0704c4 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:21.183216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:21.183851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:21.184131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:21.184402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:21.194906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:21.197004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:21.221494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:21.222516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:21.223566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:21.223566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:21.223566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:21.223566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 356/724] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:21.228634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:21.339680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:21.363564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27a7c2-42c3-4094-a64b-822fb2772541 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:21.413635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:21.414641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:21.415207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:21.415714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:21.416573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:21.417089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:21.418665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:21.434825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 369/725] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:21.440085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:21.529119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:21.579222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176403-d505-4647-81e7-466646e8b084 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:21.622637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5560691-84c2-4d5c-b69b-c5bd8607d3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:21.623309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5560691-84c2-4d5c-b69b-c5bd8607d3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:21.623589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5560691-84c2-4d5c-b69b-c5bd8607d3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:21.623865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5560691-84c2-4d5c-b69b-c5bd8607d3e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:21.635411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5560691-84c2-4d5c-b69b-c5bd8607d3e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:21.636843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5560691-84c2-4d5c-b69b-c5bd8607d3e2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d5560691-84c2-4d5c-b69b-c5bd8607d3e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:21.654802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:21.655874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:21.657246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 357/726] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:21.663822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:21.734254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9c3b2-0b23-4061-ab95-74a783294386 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:21.734619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9c3b2-0b23-4061-ab95-74a783294386 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:21.734797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9c3b2-0b23-4061-ab95-74a783294386 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:21.736146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 358/727] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:21.741998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015e5f9-9e7e-400d-b096-731ebe673a3f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:21.742281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015e5f9-9e7e-400d-b096-731ebe673a3f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:21.742554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015e5f9-9e7e-400d-b096-731ebe673a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:21.773390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9015e5f9-9e7e-400d-b096-731ebe673a3f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:21.774648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9015e5f9-9e7e-400d-b096-731ebe673a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:21.813877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef564-54f9-4841-9f17-2e1b43ffc849 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:21.850543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:21.851564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:21.852077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:21.853093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:21.854373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:21.854924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:21.856587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:21.866604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:21.870663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 359/728] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 12:04:21.871982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541401c7-45f1-40a0-b002-911251fa3b4a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:21.872349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541401c7-45f1-40a0-b002-911251fa3b4a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:21.872542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541401c7-45f1-40a0-b002-911251fa3b4a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:21.877767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 360/729] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:21.880154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10246b-f2ff-435e-a754-708e8ba8c67f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:21.880679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10246b-f2ff-435e-a754-708e8ba8c67f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:21.881063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 370/730] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 12:04:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:21.881455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10246b-f2ff-435e-a754-708e8ba8c67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:21.886994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:21.905682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1c10246b-f2ff-435e-a754-708e8ba8c67f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:21.907500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1c10246b-f2ff-435e-a754-708e8ba8c67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:21.961393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:21.991817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 361/731] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:21.994537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10246b-f2ff-435e-a754-708e8ba8c67f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:22.011256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8340373-9cf5-418b-be1a-2eed4a9e2dc6 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:22.050314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583555f-f03b-4db3-90d6-9c3ebf783230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:22.050917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583555f-f03b-4db3-90d6-9c3ebf783230 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.051227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583555f-f03b-4db3-90d6-9c3ebf783230 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.051504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583555f-f03b-4db3-90d6-9c3ebf783230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.120418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:22.123240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10246b-f2ff-435e-a754-708e8ba8c67f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:22.127754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583555f-f03b-4db3-90d6-9c3ebf783230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:22.133021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583555f-f03b-4db3-90d6-9c3ebf783230 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:22.159769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609647e-359b-4e21-b708-f22035bc9401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:22.160662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609647e-359b-4e21-b708-f22035bc9401 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.160946 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609647e-359b-4e21-b708-f22035bc9401 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.161223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609647e-359b-4e21-b708-f22035bc9401 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.161867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609647e-359b-4e21-b708-f22035bc9401 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:22.163257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609647e-359b-4e21-b708-f22035bc9401 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:22.235671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 371/732] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 12:04:22.236744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f49f-e09a-4d0c-a821-9922f70c75e2 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.237084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f49f-e09a-4d0c-a821-9922f70c75e2 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.237356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f49f-e09a-4d0c-a821-9922f70c75e2 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.242891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 372/733] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:22.243736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602f49f-e09a-4d0c-a821-9922f70c75e2 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:22.267589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 362/734] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:04:21 2022] POST /identity/v3/services => generated 251 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:22.268623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609647e-359b-4e21-b708-f22035bc9401 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:22.324335 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:22.379755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:22.400342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602f49f-e09a-4d0c-a821-9922f70c75e2 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:22.401843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609647e-359b-4e21-b708-f22035bc9401 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:22.450150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305f26-593f-4c0d-8ff2-cf0445fd4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:22.450784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305f26-593f-4c0d-8ff2-cf0445fd4b86 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.451097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305f26-593f-4c0d-8ff2-cf0445fd4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.451370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305f26-593f-4c0d-8ff2-cf0445fd4b86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.453398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:22.454367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.454655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.454960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.465426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88305f26-593f-4c0d-8ff2-cf0445fd4b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:22.467777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:22.469215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-88305f26-593f-4c0d-8ff2-cf0445fd4b86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:22.469935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:22.470861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:22.471798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:22.473827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 373/735] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:22.474796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749261dd-6f26-4cb0-9bbc-9d821d8c5c0d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.475626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749261dd-6f26-4cb0-9bbc-9d821d8c5c0d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.475626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749261dd-6f26-4cb0-9bbc-9d821d8c5c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:22.488388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:22.489703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:22.490615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:22.490615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:22.490615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:22.490615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 363/736] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:22.490926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:22.502540 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-749261dd-6f26-4cb0-9bbc-9d821d8c5c0d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:22.504452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-749261dd-6f26-4cb0-9bbc-9d821d8c5c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:22.586013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 374/737] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:22.586881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749261dd-6f26-4cb0-9bbc-9d821d8c5c0d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:22.606717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:22.619143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcfa2a-6336-46f9-b15d-6e41bd66f897 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:22.655261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e360619-f98e-4b10-a226-191c771f1607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:22.655787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e360619-f98e-4b10-a226-191c771f1607 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.656066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e360619-f98e-4b10-a226-191c771f1607 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.656336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e360619-f98e-4b10-a226-191c771f1607 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.657015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e360619-f98e-4b10-a226-191c771f1607 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:22.657305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e360619-f98e-4b10-a226-191c771f1607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:22.658656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e360619-f98e-4b10-a226-191c771f1607 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:22.666746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 364/738] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:22.667957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987c8ca-2935-4da5-b04b-23ee0bc0c829 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.668238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987c8ca-2935-4da5-b04b-23ee0bc0c829 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.668511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987c8ca-2935-4da5-b04b-23ee0bc0c829 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.669844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 365/739] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:22.674268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987c8ca-2935-4da5-b04b-23ee0bc0c829 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:22.689438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:22.708820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749261dd-6f26-4cb0-9bbc-9d821d8c5c0d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:22.745855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:22.746472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.746755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.747052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.747737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:22.748038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:22.749539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:22.763294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 375/740] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:22.764391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:22.791478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987c8ca-2935-4da5-b04b-23ee0bc0c829 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:22.826589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450f536-eb13-4621-9545-f2c9be683246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:22.827243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f536-eb13-4621-9545-f2c9be683246 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.827521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f536-eb13-4621-9545-f2c9be683246 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.828012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f536-eb13-4621-9545-f2c9be683246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.839099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450f536-eb13-4621-9545-f2c9be683246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e450f536-eb13-4621-9545-f2c9be683246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:22.843719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:22.844695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:22.845581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 366/741] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:22.846337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231980ed-3baf-49f3-b34d-647d1776f8f5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.846337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231980ed-3baf-49f3-b34d-647d1776f8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.846473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231980ed-3baf-49f3-b34d-647d1776f8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.864712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-231980ed-3baf-49f3-b34d-647d1776f8f5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:22.865901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-231980ed-3baf-49f3-b34d-647d1776f8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:22.875581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779923a-66ae-4b82-8698-4ddd32e1920f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:22.877511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:22.911845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:22.912414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:22.912674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:22.912945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:22.924202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:22.925642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:22.927400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:22.936539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 367/742] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 12:04:22.937212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231980ed-3baf-49f3-b34d-647d1776f8f5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:22.944146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:22.945277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:22.951407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:22.951407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:22.951407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:22.951407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 376/743] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:22.951407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.054558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a908cdc-cca6-4434-849b-fdd6cb94a110 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.058110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231980ed-3baf-49f3-b34d-647d1776f8f5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.090880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.091518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.091796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.092067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.092725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:23.093026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.094400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.094633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.095289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.095570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.095844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.102406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.104556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.111628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 377/744] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:23.111628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:23.119059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:23.119982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:23.120892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 368/745] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:23.121418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.132915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:23.180288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:23.225623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d005-ab1f-447d-92dd-bdcefb1859ba None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.229209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4d8fa-dd67-4776-b1b5-fac3868af23b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.261826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.262160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d22428-d21e-4d57-96e1-f3056124ee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.262424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.262703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.262803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d22428-d21e-4d57-96e1-f3056124ee5d None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.262968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.263113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d22428-d21e-4d57-96e1-f3056124ee5d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.263303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d22428-d21e-4d57-96e1-f3056124ee5d None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.263636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:23.263936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d22428-d21e-4d57-96e1-f3056124ee5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.264039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.265284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.265390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d22428-d21e-4d57-96e1-f3056124ee5d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.278056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 378/746] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:23.278695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.324869 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 369/747] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:04:23 2022] POST /identity/v3/services => generated 240 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:23.332074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d22428-d21e-4d57-96e1-f3056124ee5d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.402511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b59f2-6577-481d-8217-43c4f94c41e1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.426120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:23.426837 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:23.460554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.462097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.462551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.463440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.469095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d22428-d21e-4d57-96e1-f3056124ee5d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.478322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.480766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:23.501994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:23.503047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:23.503972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:23.503972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:23.503972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:23.503972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 379/748] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:23.503972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.519212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.521297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.521645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.521977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.523447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:23.523447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.524931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.543601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 370/749] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:23.544858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.634977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a10545-867a-40b5-921c-4437cf1212fb None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.670618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4cf25-50ef-49d9-bcb5-36cea4712056 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.678742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.679994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.680460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.681448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.691678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.693816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.705696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:23.709042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.709731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:23.709996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:23.710974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:23.712556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:23.713128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.713128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.713128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:23.713128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.713373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.720585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 380/750] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:23.720585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.737637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 371/751] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:23.737637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.758021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:23.840374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda9839-45eb-434a-a1fa-ba0af0dc32d9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.850353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd060c06-d761-4bae-a97e-66ed92f5a268 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:23.879127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.880621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.880900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.881173 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.881902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=a66a21436030409484b17f803d0e5903) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:23.882212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.883700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.890252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 381/752] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/role_assignments?user.id=a66a21436030409484b17f803d0e5903&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=f72d3388f13149b483355331380f040b => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:23.891172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.891645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:23.892517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:23.892998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:23.893488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:23.894433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:23.894924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:23.896572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:23.918505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 372/753] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:23.924366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:23.982148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:24.068849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc18e4-91c6-4904-a082-9d4e42149232 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:24.073267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:24.090954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30e811-28a6-4adf-b593-a8b097cc867a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:24.128736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7e94a-ae40-4904-9271-d1e9f7689c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:24.130363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7e94a-ae40-4904-9271-d1e9f7689c3f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.130363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7e94a-ae40-4904-9271-d1e9f7689c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.130363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7e94a-ae40-4904-9271-d1e9f7689c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.210204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7e94a-ae40-4904-9271-d1e9f7689c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:24.211745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7e94a-ae40-4904-9271-d1e9f7689c3f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:24.212546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:24.213366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.213792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.214220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.235888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:24.237576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:24.260330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:24.261539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:24.262727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:24.263337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:24.264903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 382/754] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:24.272139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:24.373345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 373/755] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 12:04:24.382245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7e94a-ae40-4904-9271-d1e9f7689c3f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:24.435841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e646bc-da00-43b3-9e55-3cdd2033ec40 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:24.472078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07f102-446a-4801-a6f4-54e8ecc2b48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:24.472690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f102-446a-4801-a6f4-54e8ecc2b48f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.472961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f102-446a-4801-a6f4-54e8ecc2b48f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.473234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f102-446a-4801-a6f4-54e8ecc2b48f None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.474077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07f102-446a-4801-a6f4-54e8ecc2b48f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:24.474386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07f102-446a-4801-a6f4-54e8ecc2b48f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:24.475855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07f102-446a-4801-a6f4-54e8ecc2b48f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:24.497974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7e94a-ae40-4904-9271-d1e9f7689c3f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:24.501329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 383/756] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:04:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:24.502665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131800e-efc5-4121-b3d6-46cc233917b8 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.502953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131800e-efc5-4121-b3d6-46cc233917b8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.503254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131800e-efc5-4121-b3d6-46cc233917b8 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.504765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 384/757] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:24.505480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131800e-efc5-4121-b3d6-46cc233917b8 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:24.514866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:24.544447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482caebc-7486-4fb9-aabe-3fd0c6ce615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:24.545445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482caebc-7486-4fb9-aabe-3fd0c6ce615b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.545934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482caebc-7486-4fb9-aabe-3fd0c6ce615b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.546403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482caebc-7486-4fb9-aabe-3fd0c6ce615b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.561416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482caebc-7486-4fb9-aabe-3fd0c6ce615b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:24.563774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482caebc-7486-4fb9-aabe-3fd0c6ce615b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:24.567710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-482caebc-7486-4fb9-aabe-3fd0c6ce615b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:24.584849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:24.585839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:24.586763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:24.586763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:24.586763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:24.591252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 374/758] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:24.592371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dfd57-cb83-4158-bb3f-701abce8ff03 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.592655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dfd57-cb83-4158-bb3f-701abce8ff03 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.592933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dfd57-cb83-4158-bb3f-701abce8ff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.613216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-579dfd57-cb83-4158-bb3f-701abce8ff03 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:24.614799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-579dfd57-cb83-4158-bb3f-701abce8ff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:24.627034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131800e-efc5-4121-b3d6-46cc233917b8 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:24.671032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:24.671713 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.671995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.672264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.689000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:24.691238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:24.703250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 375/759] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:24.704660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dfd57-cb83-4158-bb3f-701abce8ff03 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:24.711172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:24.712374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:24.713407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:24.713407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:24.713407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:24.713892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 385/760] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:24.718479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:24.801033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:24.836784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dfd57-cb83-4158-bb3f-701abce8ff03 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:24.849339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dfdab-35ec-4d88-858d-708d89fde2d4 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:24.870862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:24.882728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:24.883581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.884633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.885273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.885680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:24.886308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:24.886425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:24.886818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:24.887036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:24.887319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:24.888071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:24.888644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:24.889360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:24.890557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:24.903264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 376/761] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:24.904263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:24.908591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 386/762] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:24.909072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.021409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85073052-df21-4268-920b-fb04597a3fa4 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.021872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa013b2-5c85-45f5-93d7-a0db9155f312 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.039887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:25.058191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.058617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.059283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.059564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.059832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.059947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.060419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.060524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.061038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.061784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.068464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.070945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:25.071488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:25.072509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:25.073394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:25.078126 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 377/763] 10.222.0.30 () {56 vars in 1129 bytes} [Sun Aug 7 12:04:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:25.079012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:25.087629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:25.089074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:25.090022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:25.090022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:25.090022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:25.090022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:25.090022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:25.090022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:25.090022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:25.090022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:25.092476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 387/764] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:25.093490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.144786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:25.202501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44221f-a988-4351-b0ea-8df3ac52b816 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.215297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316c2d2-7036-417c-acee-43dbcde13dc7 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.240328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.240963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.241173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.241449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.249652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.251323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.252550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.253410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.253891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.254364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.255241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:25.255750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.257324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.262723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 388/765] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:25.263490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:25.264490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:25.265402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:25.265402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:25.265402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:25.265402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:25.265402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:25.265402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:25.265402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:25.265402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:25.266694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.269486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 378/766] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:25.270312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.313698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:25.371685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc730506-c808-4d61-b3a8-55e4a3502cc2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.384686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a30206-b5eb-4da3-afd5-0405f9415a59 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.396621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:25.407502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.408217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.408558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.410472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.410472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:25.410472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.411630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.424249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.429637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.429764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 389/767] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:25.429764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.429934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.429934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.429934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:25.429934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.429934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.447392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 379/768] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:25.448602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.534464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4ec55-359a-46e2-9af8-915505a6cc99 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.536064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:25.565157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d24311-bb09-4b7a-a88d-5232934c481a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.570175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.570785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.576302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.576302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.576302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:25.576302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.576302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.586486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 390/769] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:25.587414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.607848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedb65c-bc54-4490-a196-68c14b19b50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.609318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedb65c-bc54-4490-a196-68c14b19b50c None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.609806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedb65c-bc54-4490-a196-68c14b19b50c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.610294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedb65c-bc54-4490-a196-68c14b19b50c None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/cb2075e9fff3459881c13f81456ab198/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.613384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedb65c-bc54-4490-a196-68c14b19b50c None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=cb2075e9fff3459881c13f81456ab198, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.615748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedb65c-bc54-4490-a196-68c14b19b50c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.681728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 380/770] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:25 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/cb2075e9fff3459881c13f81456ab198/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:25.683295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a921-53a9-40d4-98de-f4a9ff7d580f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.683747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a921-53a9-40d4-98de-f4a9ff7d580f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.684179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a921-53a9-40d4-98de-f4a9ff7d580f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.685945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 381/771] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:25.686944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a921-53a9-40d4-98de-f4a9ff7d580f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.702349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f3b7f-2524-4be8-8e9b-8d66c8c0c5e6 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.733649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:25.752531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b36724-e0d8-4765-a153-a358cbe49bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.753807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b36724-e0d8-4765-a153-a358cbe49bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.754365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b36724-e0d8-4765-a153-a358cbe49bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.755479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b36724-e0d8-4765-a153-a358cbe49bf1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.756545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b36724-e0d8-4765-a153-a358cbe49bf1 None admin] RBAC: Adding query filter params (interface=public, service_id=ad814d813b3a4006b83416241d2d11c6, region_id=RegionOne) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:25.757237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b36724-e0d8-4765-a153-a358cbe49bf1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.760851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b36724-e0d8-4765-a153-a358cbe49bf1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.773471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 391/772] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/endpoints?service_id=ad814d813b3a4006b83416241d2d11c6&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:25.775393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda4dbc-ca4f-4d15-be9c-d0665a42b5df None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.775918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda4dbc-ca4f-4d15-be9c-d0665a42b5df None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.776367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda4dbc-ca4f-4d15-be9c-d0665a42b5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.799489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-adda4dbc-ca4f-4d15-be9c-d0665a42b5df None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:25.800765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-adda4dbc-ca4f-4d15-be9c-d0665a42b5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:25.825588 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:25.834655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a921-53a9-40d4-98de-f4a9ff7d580f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:25.881983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 392/773] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:25.883043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda4dbc-ca4f-4d15-be9c-d0665a42b5df None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:25.887567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:25.888170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:25.888445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:25.888713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:25.889443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=87a5db9a38544dc189b0c0d7bb205d83) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:25.889747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:25.891183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:25.900676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 382/774] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/role_assignments?user.id=87a5db9a38544dc189b0c0d7bb205d83&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:25.906955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:26.026288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106912c-edfb-4e8e-bb46-b0fa1ed9b830 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:26.029177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:26.066748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ce647-2ca8-42a8-921c-7b3eff7544b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:26.067205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda4dbc-ca4f-4d15-be9c-d0665a42b5df None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:26.067433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ce647-2ca8-42a8-921c-7b3eff7544b3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.067708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ce647-2ca8-42a8-921c-7b3eff7544b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.067973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ce647-2ca8-42a8-921c-7b3eff7544b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.068612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ce647-2ca8-42a8-921c-7b3eff7544b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:26.071248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ce647-2ca8-42a8-921c-7b3eff7544b3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-713ce647-2ca8-42a8-921c-7b3eff7544b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:26.083611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:26.084863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:26.086239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 383/775] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:26.090317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef48b-402e-4010-81e8-9c440cb24276 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.090317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef48b-402e-4010-81e8-9c440cb24276 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.090317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef48b-402e-4010-81e8-9c440cb24276 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.090971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 384/776] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:26.091739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef48b-402e-4010-81e8-9c440cb24276 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:26.122693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d4f76-f9e8-4186-833b-7ce81dd27ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:26.124056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d4f76-f9e8-4186-833b-7ce81dd27ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.124341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d4f76-f9e8-4186-833b-7ce81dd27ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.124615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d4f76-f9e8-4186-833b-7ce81dd27ac4 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.125788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d4f76-f9e8-4186-833b-7ce81dd27ac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:26.126607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d4f76-f9e8-4186-833b-7ce81dd27ac4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:26.132813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 393/777] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:04:25 2022] GET /identity/v3/services => generated 772 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:26.134987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71925605-d7c2-4157-aa22-e47b0445f24b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.135380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71925605-d7c2-4157-aa22-e47b0445f24b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.135660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71925605-d7c2-4157-aa22-e47b0445f24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.155758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-71925605-d7c2-4157-aa22-e47b0445f24b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:26.156964 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-71925605-d7c2-4157-aa22-e47b0445f24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:26.173621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:26.235290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 394/778] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:26.260614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71925605-d7c2-4157-aa22-e47b0445f24b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:26.278183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef48b-402e-4010-81e8-9c440cb24276 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:26.326566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:26.329557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.330230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.331327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.332369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:26.332816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:26.339379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:26.349240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 385/779] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:26.355459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:26.393014 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:26.423989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71925605-d7c2-4157-aa22-e47b0445f24b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:26.469495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d37427-eba3-452e-a262-5e07c55f6194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:26.470542 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d37427-eba3-452e-a262-5e07c55f6194 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.471100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d37427-eba3-452e-a262-5e07c55f6194 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.471576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d37427-eba3-452e-a262-5e07c55f6194 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.480031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b5fc-c497-460c-9c69-622f26d55cb8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:26.483589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d37427-eba3-452e-a262-5e07c55f6194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-30d37427-eba3-452e-a262-5e07c55f6194 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:26.488765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:26.489966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:26.490824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:26.493555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 395/780] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:26.499563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d37427-eba3-452e-a262-5e07c55f6194 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:26.512918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:26.519596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:26.520449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.520972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.521254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.521893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] RBAC: Adding query filter params (type=network) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:26.526527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:26.526527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:26.534227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 386/781] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:26.539924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:26.679369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:26.690768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d37427-eba3-452e-a262-5e07c55f6194 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:26.698152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ff1d1-31bc-4c2f-a8fc-d8c2a9714d2a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:26.761881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:26.762482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.762757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.763038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.763711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:26.764009 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:26.765383 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:26.769057 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:26.769672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.769948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.770221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.770983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] RBAC: Adding query filter params (interface=public, service_id=95c5e15171f448c0b1d5d8360da4b05e, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:26.771314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:26.773019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:26.788069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 396/782] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:26.792164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 387/783] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 12:04:26 2022] GET /identity/v3/endpoints?service_id=95c5e15171f448c0b1d5d8360da4b05e&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:26.794667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:26.797387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:26.889549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:26.921432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012cf9e-44cd-43e4-a175-40c098470cb9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:26.957429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:26.957980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:26.958259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:26.958531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:26.970005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:26.971512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:26.986567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:26.987603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:26.989370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:26.989370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:26.989370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:26.989370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 397/784] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:26.993745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:27.052980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaabd4c-ac81-49bb-bb44-7a31ca22a6e7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:27.100056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c3f4b-9a6d-4934-980e-b3dd6b0493fa None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:27.135390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:27.143688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:27.144481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:27.145883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:27.146443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:27.147698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:27.149005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:27.149847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97697e1-6295-4e76-b9cf-f3437d653f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:27.151478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:27.152327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97697e1-6295-4e76-b9cf-f3437d653f21 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:27.156628 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97697e1-6295-4e76-b9cf-f3437d653f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:27.156628 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97697e1-6295-4e76-b9cf-f3437d653f21 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:27.156628 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97697e1-6295-4e76-b9cf-f3437d653f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:27.157918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97697e1-6295-4e76-b9cf-f3437d653f21 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:27.170872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 388/785] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:04:26 2022] GET /identity/v3/services => generated 772 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:27.174274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 398/786] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:27.180205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97697e1-6295-4e76-b9cf-f3437d653f21 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:27.210038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:27.335272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97697e1-6295-4e76-b9cf-f3437d653f21 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:27.375944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:27.376571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:27.376850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:27.377212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:27.386914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:27.388383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:27.402783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:27.403797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:27.405347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 389/787] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:27.409791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:27.530178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ee1b1-cd09-4ccc-a195-5db3dd5cdb78 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:27.567932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:27.568553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:27.568837 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:27.569174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:27.569853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:27.570155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:27.571805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:27.590647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 399/788] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 12:04:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:27.596029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:27.730525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:27.756542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c7f8-de3e-4d06-a672-2a585cb780f3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:27.798391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:27.798936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:27.799244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:27.799516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:27.810643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:27.812179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:27.827036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:27.828164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:27.829419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:27.829419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:27.829419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:27.829419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 390/789] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:27.833845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:27.903380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:27.967350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3693f-435c-470e-9e69-51e7169ca67b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:28.018144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4da11b-9ec4-4c41-8784-4daefb8b9846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:28.018754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4da11b-9ec4-4c41-8784-4daefb8b9846 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.019049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4da11b-9ec4-4c41-8784-4daefb8b9846 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.019328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4da11b-9ec4-4c41-8784-4daefb8b9846 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.019976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4da11b-9ec4-4c41-8784-4daefb8b9846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:28.020275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4da11b-9ec4-4c41-8784-4daefb8b9846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:28.021771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4da11b-9ec4-4c41-8784-4daefb8b9846 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:28.041972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 400/790] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:28.047281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:28.146948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3c09d-4327-4477-9408-4717e5b36d5c None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.147497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3c09d-4327-4477-9408-4717e5b36d5c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.147497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3c09d-4327-4477-9408-4717e5b36d5c None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.149088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 401/791] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:28.168431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccd14a-c777-4952-831e-716c8cf5a21e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.168726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccd14a-c777-4952-831e-716c8cf5a21e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.168983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccd14a-c777-4952-831e-716c8cf5a21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.203375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-feccd14a-c777-4952-831e-716c8cf5a21e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:28.204563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-feccd14a-c777-4952-831e-716c8cf5a21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:28.215958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c336d65-61c8-48db-b407-6d556c1c1b56 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:28.279694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 402/792] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:28.287344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:28.303577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:28.304046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccd14a-c777-4952-831e-716c8cf5a21e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:28.304242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.304511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.304784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.318007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:28.319496 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:28.335964 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:28.342087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:28.343332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:28.344257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:28.344257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:28.344257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:28.344257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:28.344257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:28.344257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:28.344257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:28.344257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:28.349232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 391/793] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:28.357185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:28.407499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccd14a-c777-4952-831e-716c8cf5a21e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:28.443869 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a05ff-1bcc-45eb-9e78-1e26110bbfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:28.444521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a05ff-1bcc-45eb-9e78-1e26110bbfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.444903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a05ff-1bcc-45eb-9e78-1e26110bbfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.445179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a05ff-1bcc-45eb-9e78-1e26110bbfc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.456809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a05ff-1bcc-45eb-9e78-1e26110bbfc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-c74a05ff-1bcc-45eb-9e78-1e26110bbfc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:28.461988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:28.462945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:28.464461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 403/794] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:28.465175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898e26-4d29-4a11-b14f-f33009561f06 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.465175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898e26-4d29-4a11-b14f-f33009561f06 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.465502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898e26-4d29-4a11-b14f-f33009561f06 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.467044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 404/795] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:28.469171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df898e26-4d29-4a11-b14f-f33009561f06 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:28.518488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06dc77-fdb0-4df8-aeaa-8977284d11d2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:28.554385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1fe63-77da-49ef-9ba0-2adeb42fd5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:28.554893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1fe63-77da-49ef-9ba0-2adeb42fd5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.555218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1fe63-77da-49ef-9ba0-2adeb42fd5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.555489 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1fe63-77da-49ef-9ba0-2adeb42fd5a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.556417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1fe63-77da-49ef-9ba0-2adeb42fd5a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:28.556724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1fe63-77da-49ef-9ba0-2adeb42fd5a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:28.558234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1fe63-77da-49ef-9ba0-2adeb42fd5a5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:28.571142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 392/796] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:28.573259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe98a1-e8e1-448d-8214-ec8913bddaf2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.573543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe98a1-e8e1-448d-8214-ec8913bddaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.573812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe98a1-e8e1-448d-8214-ec8913bddaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.592011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-61fe98a1-e8e1-448d-8214-ec8913bddaf2 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:28.593105 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-61fe98a1-e8e1-448d-8214-ec8913bddaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:28.600036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:28.613735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df898e26-4d29-4a11-b14f-f33009561f06 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:28.646219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:28.656068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:28.656948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.657373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.657941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.658799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:28.659310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:28.660888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:28.670657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 405/797] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:28.671948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:28.675098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 393/798] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:28.676125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe98a1-e8e1-448d-8214-ec8913bddaf2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:28.796085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362f8d2-7641-4e76-bbb6-6104400f6e71 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:28.806204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe98a1-e8e1-448d-8214-ec8913bddaf2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:28.831377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3603f-a590-4213-9d78-eb6e8876bc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:28.831957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3603f-a590-4213-9d78-eb6e8876bc59 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.832240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3603f-a590-4213-9d78-eb6e8876bc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.832517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3603f-a590-4213-9d78-eb6e8876bc59 None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/a66a21436030409484b17f803d0e5903/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.834484 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3603f-a590-4213-9d78-eb6e8876bc59 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=a66a21436030409484b17f803d0e5903, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:28.835957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3603f-a590-4213-9d78-eb6e8876bc59 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:28.842465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:28.843120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.843397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.843665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.853555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:28.854973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:28.869687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:28.870829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:28.871797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:28.871797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:28.871797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:28.871797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 394/799] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:28.871797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:28.903029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:28.916208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 406/800] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:28 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/a66a21436030409484b17f803d0e5903/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:28.917900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca99645-2858-4781-bb07-d420614bd960 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:28.918355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca99645-2858-4781-bb07-d420614bd960 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:28.918766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca99645-2858-4781-bb07-d420614bd960 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:28.920314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 407/801] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:28.921302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca99645-2858-4781-bb07-d420614bd960 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:28.945126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:28.994243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303a7f8-6acb-4b02-b150-a72622943a74 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.041678 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aee538-0427-40b5-ac02-dd1de16da8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.042820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca99645-2858-4781-bb07-d420614bd960 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.043223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee538-0427-40b5-ac02-dd1de16da8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.047579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee538-0427-40b5-ac02-dd1de16da8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.047579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee538-0427-40b5-ac02-dd1de16da8c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.057737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aee538-0427-40b5-ac02-dd1de16da8c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-18aee538-0427-40b5-ac02-dd1de16da8c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:29.062899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:29.064036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:29.064912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:29.067743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 395/802] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:29.069388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e1ae-4ef6-4078-8c7f-620b9e36cfdb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.069897 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e1ae-4ef6-4078-8c7f-620b9e36cfdb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.070377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e1ae-4ef6-4078-8c7f-620b9e36cfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.082495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.083138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.083417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.083691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.084340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:29.084643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:29.086088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:29.093272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4b52e1ae-4ef6-4078-8c7f-620b9e36cfdb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:29.094804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4b52e1ae-4ef6-4078-8c7f-620b9e36cfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:29.100220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 408/803] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:29.101139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.184236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 396/804] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:29.185410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e1ae-4ef6-4078-8c7f-620b9e36cfdb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.185883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:29.231333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:29.233619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d4213-61ee-45e2-9832-2fd9c4e63faf None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.275272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.276388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.276971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.277566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.278446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:29.278984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:29.280865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:29.293593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 409/805] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:29.295518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.338835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e1ae-4ef6-4078-8c7f-620b9e36cfdb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.389990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.390703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.391061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.391338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.400458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:29.401890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:29.421765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:29.422984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:29.424390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:29.427091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 397/806] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:29.428168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.440578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87c163-83f8-4711-b1b4-8d08dcaa1d3a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.495247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.496444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.496902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.497550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.498908 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:29.500536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:29.505482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:29.516475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:29.517535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 12:04:29.518466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:29.526271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 410/807] 10.222.0.30 () {56 vars in 1129 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:29.526271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.549821 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:29.576730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e232e-4a53-4225-85fd-dba49892e055 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.635740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.637844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.638563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.639785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.660441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915f7ce-dd14-4a34-84ef-0221b09d8861 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.662210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:29.664432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:29.680695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:29.681857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:29.682766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:29.682766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:29.682766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:29.682766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 398/808] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:29.683536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.709002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.709518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.709800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.710135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.711258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:29.711387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:29.713925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:29.736855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 411/809] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:29.738134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.807068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:29.807427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7c9ec-837c-4dec-bdb9-4cc1cbeba304 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.845433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15049e23-bf05-4eb4-a330-529cd055a244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.846045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15049e23-bf05-4eb4-a330-529cd055a244 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.846327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15049e23-bf05-4eb4-a330-529cd055a244 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.846604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15049e23-bf05-4eb4-a330-529cd055a244 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.847260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15049e23-bf05-4eb4-a330-529cd055a244 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:29.847613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15049e23-bf05-4eb4-a330-529cd055a244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:29.848977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15049e23-bf05-4eb4-a330-529cd055a244 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:29.850146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:29.854345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 399/810] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:29.861615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618dfa-370a-472c-9c05-fb52d922e94c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.861615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618dfa-370a-472c-9c05-fb52d922e94c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.861615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618dfa-370a-472c-9c05-fb52d922e94c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.861615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 400/811] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:29.861615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37618dfa-370a-472c-9c05-fb52d922e94c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.868982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f5854-159b-463d-8330-f79e573d68aa None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:29.908282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:29.908902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:29.909178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:29.909454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:29.910095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:29.910398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:29.911809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:29.923602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 412/812] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:29.924430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:29.986218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37618dfa-370a-472c-9c05-fb52d922e94c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.029855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1c7cf-6439-4316-88b3-87562ba321e3 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.034735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92488af5-30db-4181-8ede-84c8878bcc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.035703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92488af5-30db-4181-8ede-84c8878bcc59 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.036195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92488af5-30db-4181-8ede-84c8878bcc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.036757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92488af5-30db-4181-8ede-84c8878bcc59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.049208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:30.067920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5b659-cabe-4b09-b572-85db7a97065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.068524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b659-cabe-4b09-b572-85db7a97065e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.068808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b659-cabe-4b09-b572-85db7a97065e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.069079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b659-cabe-4b09-b572-85db7a97065e None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.069706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5b659-cabe-4b09-b572-85db7a97065e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:30.070010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5b659-cabe-4b09-b572-85db7a97065e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.072047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5b659-cabe-4b09-b572-85db7a97065e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.081111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 413/813] 10.222.0.30 () {56 vars in 1124 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:30.081111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbd11c-de04-45eb-a5f2-ec2f857ac6bc None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.081111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbd11c-de04-45eb-a5f2-ec2f857ac6bc None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.081111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbd11c-de04-45eb-a5f2-ec2f857ac6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.097911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-62dbd11c-de04-45eb-a5f2-ec2f857ac6bc None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:30.099308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-62dbd11c-de04-45eb-a5f2-ec2f857ac6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:30.119533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:30.124379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92488af5-30db-4181-8ede-84c8878bcc59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.125926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92488af5-30db-4181-8ede-84c8878bcc59 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.178044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 414/814] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:30.179173 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbd11c-de04-45eb-a5f2-ec2f857ac6bc None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:30.241155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 401/815] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:04:30.246813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92488af5-30db-4181-8ede-84c8878bcc59 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:30.294987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbd11c-de04-45eb-a5f2-ec2f857ac6bc None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.310014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:30.332455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.333359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.333641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.333910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.341590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.343836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.348745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:30.352442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92488af5-30db-4181-8ede-84c8878bcc59 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:30.356067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:30.357067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:30.358054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 415/816] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:04:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:30.358681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:30.395637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f7d70-d1db-4afc-a254-a2e13294b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.396998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f7d70-d1db-4afc-a254-a2e13294b953 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.396998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f7d70-d1db-4afc-a254-a2e13294b953 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.396998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f7d70-d1db-4afc-a254-a2e13294b953 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.397722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f7d70-d1db-4afc-a254-a2e13294b953 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.399182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f7d70-d1db-4afc-a254-a2e13294b953 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.462780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc650ff7-f82b-499d-ae9c-64af11245321 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.496145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.496700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.496983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.497292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] PATH_INFO: `/v3/services/network` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.497876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.499285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:30.507817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:30.509772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:30.510686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 416/817] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:30 2022] GET /identity/v3/services/network => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:30.512003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:30.527104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 402/818] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:04:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:30.527104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f7d70-d1db-4afc-a254-a2e13294b953 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:30.562487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:30.657985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e14368-39bc-47cc-8f87-69c1947937dd None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.673798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f7d70-d1db-4afc-a254-a2e13294b953 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.701966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.702601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.702884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.703194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.715453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.716933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.722661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.723524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.723984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.724433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.725897 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:30.726389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.729213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:30.733306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:30.734317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:30.735341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:30.735341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:30.735341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:30.738975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 417/819] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:30.740138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:30.758006 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 403/820] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 12:04:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:30.760239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:30.775054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:30.853816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:30.867778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccad2d3-f802-457d-a55e-7a816a9c1a6b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.907447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.908293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.908718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.909164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.909280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240d5f5-25dd-4e7d-add4-a8dc9c050155 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:30.910034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] RBAC: Adding query filter params (name=network) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:30.910485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.912182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.924497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 418/821] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:30.925684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:30.953507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:30.954116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:30.954397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:30.954671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:30.955612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:30.956683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:30.958303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:30.975195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 404/822] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:30.976687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.043767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c64688-bc96-4167-9be9-6bb2555e208e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.068488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:31.086550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.088101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.088519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.088950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.103453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:31.105157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:31.110872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f136dc1-ed83-438f-8366-a8b787becf81 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:31.125536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:31.126737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:31.127902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:31.127902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:31.127902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:31.131230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 419/823] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:31.132046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.136405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:31.151404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9fc2-e6c1-4f4c-8633-87ab3b65573f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.152108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9fc2-e6c1-4f4c-8633-87ab3b65573f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.152289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9fc2-e6c1-4f4c-8633-87ab3b65573f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.152568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9fc2-e6c1-4f4c-8633-87ab3b65573f None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.153280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9fc2-e6c1-4f4c-8633-87ab3b65573f None admin] RBAC: Adding query filter params (type=network) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:31.153590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9fc2-e6c1-4f4c-8633-87ab3b65573f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:31.155117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9fc2-e6c1-4f4c-8633-87ab3b65573f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:31.160048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 405/824] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:31.161157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa316f-c0a3-41b1-a0b1-c5c62b6c5f5d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.161449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa316f-c0a3-41b1-a0b1-c5c62b6c5f5d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.161725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa316f-c0a3-41b1-a0b1-c5c62b6c5f5d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.167165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 406/825] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:31.167165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa316f-c0a3-41b1-a0b1-c5c62b6c5f5d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.244758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf3c77-07bc-4e19-84f4-4687068db7e1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.281066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa316f-c0a3-41b1-a0b1-c5c62b6c5f5d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.282308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.282943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.283322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.283603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.290444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:31.292116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:31.303573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:31.304900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:31.306333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:31.306333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:31.306333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:31.306333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:31.306333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:31.306333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:31.306333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:31.306333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:31.307260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 420/826] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:31.307943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.320421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:31.326633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a3e33-9e09-4075-a37e-b47fc965f74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.327429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a3e33-9e09-4075-a37e-b47fc965f74a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.327641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a3e33-9e09-4075-a37e-b47fc965f74a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.327909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a3e33-9e09-4075-a37e-b47fc965f74a None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.328661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a3e33-9e09-4075-a37e-b47fc965f74a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:31.328960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a3e33-9e09-4075-a37e-b47fc965f74a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:31.330301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a3e33-9e09-4075-a37e-b47fc965f74a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:31.347710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 407/827] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:31.349006 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4634571-ccdc-4152-bfa5-ae9374c2085f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.349289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4634571-ccdc-4152-bfa5-ae9374c2085f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.349559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4634571-ccdc-4152-bfa5-ae9374c2085f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.369569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a4634571-ccdc-4152-bfa5-ae9374c2085f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:31.370811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a4634571-ccdc-4152-bfa5-ae9374c2085f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:31.374702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:31.415081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9217dd3-87b2-4c8b-8ddd-a084ee80f9da None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.443053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 408/828] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:31.443053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4634571-ccdc-4152-bfa5-ae9374c2085f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.460278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3b62f-4a6e-43c4-82a4-73ccb3802642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.461218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3b62f-4a6e-43c4-82a4-73ccb3802642 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.461695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3b62f-4a6e-43c4-82a4-73ccb3802642 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.462704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3b62f-4a6e-43c4-82a4-73ccb3802642 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.463563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3b62f-4a6e-43c4-82a4-73ccb3802642 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:31.465288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3b62f-4a6e-43c4-82a4-73ccb3802642 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:31.546157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4634571-ccdc-4152-bfa5-ae9374c2085f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.547667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 421/829] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:04:31 2022] POST /identity/v3/endpoints => generated 337 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:31.548574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3b62f-4a6e-43c4-82a4-73ccb3802642 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.585443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:31.615692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.616326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.616607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.616877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.617580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:31.617884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:31.619955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:31.639036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 409/830] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:31.639036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.697522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3b62f-4a6e-43c4-82a4-73ccb3802642 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.723935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:31.749231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.749893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.750179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.750456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.762348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:31.763942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:31.778789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:31.779812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:31.780810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:31.780810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:31.780810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:31.780810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:31.780810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:31.780810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:31.780810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:31.780810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:31.783174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 422/831] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:31.784256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.787891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb116b-e54f-4c9e-9769-2adadf5a21b9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.831874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4869a-bc62-4301-888e-befc92bd2615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.832590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4869a-bc62-4301-888e-befc92bd2615 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.832937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4869a-bc62-4301-888e-befc92bd2615 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.833299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4869a-bc62-4301-888e-befc92bd2615 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.844233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4869a-bc62-4301-888e-befc92bd2615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6ad4869a-bc62-4301-888e-befc92bd2615 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:31.849264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:31.850225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 410/832] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:31.851141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4869a-bc62-4301-888e-befc92bd2615 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:31.895479 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:31.904815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a6641-736c-4828-a9a2-c10effa0f915 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.941859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:31.942457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:31.942744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:31.943032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:31.943805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=cb2075e9fff3459881c13f81456ab198) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:31.944118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:31.945553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:31.958739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4869a-bc62-4301-888e-befc92bd2615 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:31.963374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 423/833] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:31 2022] GET /identity/v3/role_assignments?user.id=cb2075e9fff3459881c13f81456ab198&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=f72d3388f13149b483355331380f040b => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:31.971141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:32.006070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:32.006797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.007127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.007402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.008106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:32.008413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:32.010244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:32.029743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:32.031474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 411/834] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:32.032426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:32.098479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd9f6-1fb2-4f03-a838-145b93d488da None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:32.155374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:32.158368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:32.159224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.159505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.159776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.160452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:32.160755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:32.161468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44fb11-c522-450c-98ad-71244b3797d2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:32.162140 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:32.176044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 424/835] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:32.177709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:32.204758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640a61f-180d-4482-8443-2943b39f2cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:32.205512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a61f-180d-4482-8443-2943b39f2cd9 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.205942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a61f-180d-4482-8443-2943b39f2cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.206292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a61f-180d-4482-8443-2943b39f2cd9 None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/87a5db9a38544dc189b0c0d7bb205d83/roles/2e4b771225e74dfc974d2ba9b872c77f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.223142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640a61f-180d-4482-8443-2943b39f2cd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=87a5db9a38544dc189b0c0d7bb205d83, role_id=2e4b771225e74dfc974d2ba9b872c77f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:32.224675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640a61f-180d-4482-8443-2943b39f2cd9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:32.295087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b90ce-5735-4ff3-8c92-6e8d6a5af462 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:32.297730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 412/836] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:32 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/87a5db9a38544dc189b0c0d7bb205d83/roles/2e4b771225e74dfc974d2ba9b872c77f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:32.309998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:32.342097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:32.342845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.343170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.343507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.356188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:32.357661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:32.375438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:32.376509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:32.377451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:32.377451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:32.377451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:32.377451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 425/837] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:32.384680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640a61f-180d-4482-8443-2943b39f2cd9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:32.511436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640a61f-180d-4482-8443-2943b39f2cd9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:32.547202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196887b9-b128-46c7-a241-b46fd4f9f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:32.547806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196887b9-b128-46c7-a241-b46fd4f9f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.548084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196887b9-b128-46c7-a241-b46fd4f9f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.548353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196887b9-b128-46c7-a241-b46fd4f9f5e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.548995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196887b9-b128-46c7-a241-b46fd4f9f5e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:32.549290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196887b9-b128-46c7-a241-b46fd4f9f5e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:32.550641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196887b9-b128-46c7-a241-b46fd4f9f5e6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:32.563388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 413/838] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:32.568597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:32.673197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aefe48-8360-47aa-88b8-3418e62935f9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:32.698170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adde77-5e98-44a5-9af0-d1540660de7d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.698514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adde77-5e98-44a5-9af0-d1540660de7d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.698712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adde77-5e98-44a5-9af0-d1540660de7d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.700220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 414/839] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:32.705170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844e61d-c7f7-45a1-bd18-f237c84c417a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.705457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844e61d-c7f7-45a1-bd18-f237c84c417a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.705744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844e61d-c7f7-45a1-bd18-f237c84c417a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.714744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:32.715425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.715702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.715972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.724033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:32.725486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:32.726650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2844e61d-c7f7-45a1-bd18-f237c84c417a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:32.728626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2844e61d-c7f7-45a1-bd18-f237c84c417a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:32.731214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:32.738251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:32.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:32.740526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 426/840] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:32.745168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:32.765333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:32.822596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 415/841] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:32.844192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844e61d-c7f7-45a1-bd18-f237c84c417a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:32.861306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f5b0-195e-475d-8ba4-a0e136e9a546 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:32.899984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:32.901176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.901655 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.902138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.903207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:32.903729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:32.905398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:32.925360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 427/842] 10.222.0.30 () {56 vars in 1135 bytes} [Sun Aug 7 12:04:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:32.931995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:32.955401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844e61d-c7f7-45a1-bd18-f237c84c417a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:32.992051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:32.992691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:32.993120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:32.993906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:32.995322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:32.996642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:33.005720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:33.006714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:33.007605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:33.011674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 416/843] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:33.012667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:33.016117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:33.022983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:33.047154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d0454-a265-442e-85d3-2a8e790eadee None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:33.084715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68923e68-508f-4d10-9df5-31fb9b36ef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:33.085623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68923e68-508f-4d10-9df5-31fb9b36ef82 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:33.086100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68923e68-508f-4d10-9df5-31fb9b36ef82 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:33.086936 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68923e68-508f-4d10-9df5-31fb9b36ef82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:33.129584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b118703-af4d-4dea-b312-6cf185b37a30 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:33.164189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68923e68-508f-4d10-9df5-31fb9b36ef82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:33.166502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68923e68-508f-4d10-9df5-31fb9b36ef82 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:33.168332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:33.169057 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:33.169347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:33.169626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:33.170878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:33.170878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:33.172045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:33.183098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 417/844] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:33.184207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:33.255025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:33.262836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:33.285197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 428/845] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:32 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 12:04:33.286170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68923e68-508f-4d10-9df5-31fb9b36ef82 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:33.311329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fdab2-daa9-4f30-94c6-dcff72b7d97b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:33.359482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:33.360357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:33.360794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:33.361242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:33.362018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:33.362471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:33.364777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:33.377021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 418/846] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:33.397818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68923e68-508f-4d10-9df5-31fb9b36ef82 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:33.436863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:33.437425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:33.437699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:33.437966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:33.450672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:33.452174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:33.468690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:33.469752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:33.470676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:33.470676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:33.470676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:33.470676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 429/847] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:33.475764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:33.601118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8caef3-7962-40af-8d01-73b88104eb14 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:33.634257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:33.640410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:33.641022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:33.641301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:33.641625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:33.642276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:33.642579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:33.644085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:33.658298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 419/848] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:33.663514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:33.694751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:33.769246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b9ecc-2ecc-425b-8983-bab91cebe71b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:33.804590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:33.805217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:33.806102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:33.806391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:33.813370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:33.814825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:33.826291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:33.827244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:33.828804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 430/849] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:33.836094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:33.942676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2b6e8-e801-493f-aa3f-a2968446a6b6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:33.979272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128da7e-87ea-4bc1-a0c6-abc85e342344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:33.979788 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128da7e-87ea-4bc1-a0c6-abc85e342344 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:33.980064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128da7e-87ea-4bc1-a0c6-abc85e342344 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:33.980333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128da7e-87ea-4bc1-a0c6-abc85e342344 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:33.981018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128da7e-87ea-4bc1-a0c6-abc85e342344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:33.981318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128da7e-87ea-4bc1-a0c6-abc85e342344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:33.982727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128da7e-87ea-4bc1-a0c6-abc85e342344 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:33.995401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 420/850] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:34.001302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:34.094394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:34.106941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6386e4-a0c2-4940-9dc3-0e3dc23e1ee4 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:34.122629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc5ee-fea1-40bc-8964-db9124a4b88a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.123314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc5ee-fea1-40bc-8964-db9124a4b88a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.123813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc5ee-fea1-40bc-8964-db9124a4b88a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.129452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 421/851] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:34.130934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aa2f7-04dd-4ad2-97bb-cf9e81ae2a9e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.131480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aa2f7-04dd-4ad2-97bb-cf9e81ae2a9e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.132048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aa2f7-04dd-4ad2-97bb-cf9e81ae2a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.144932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:34.145533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.145809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.146078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.146793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=a66a21436030409484b17f803d0e5903) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:34.147123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:34.149356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:34.153505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a12aa2f7-04dd-4ad2-97bb-cf9e81ae2a9e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:34.154943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a12aa2f7-04dd-4ad2-97bb-cf9e81ae2a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:34.155828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 431/852] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:34 2022] GET /identity/v3/role_assignments?user.id=a66a21436030409484b17f803d0e5903&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=f72d3388f13149b483355331380f040b => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:34.214438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:34.237928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 422/853] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:34.262576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:34.375604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae55795-0688-41a5-9595-c3f40f2c4287 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:34.411628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966913d5-8000-4097-b90a-0885bf1b372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:34.412928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966913d5-8000-4097-b90a-0885bf1b372e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.413355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966913d5-8000-4097-b90a-0885bf1b372e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.413793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966913d5-8000-4097-b90a-0885bf1b372e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.414522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966913d5-8000-4097-b90a-0885bf1b372e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:34.416771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966913d5-8000-4097-b90a-0885bf1b372e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-966913d5-8000-4097-b90a-0885bf1b372e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:34.425906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:34.427133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:34.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:34.430862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 432/854] 10.222.0.30 () {56 vars in 1132 bytes} [Sun Aug 7 12:04:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:34.437122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aa2f7-04dd-4ad2-97bb-cf9e81ae2a9e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:34.488815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5820e-e709-49ef-8793-11c3ee4228d4 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.489020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5820e-e709-49ef-8793-11c3ee4228d4 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.489532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5820e-e709-49ef-8793-11c3ee4228d4 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.492148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 433/855] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:34.496462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832ea13-72cc-44cf-88b2-8fb8541a409b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.496742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832ea13-72cc-44cf-88b2-8fb8541a409b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.497015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832ea13-72cc-44cf-88b2-8fb8541a409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.520726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8832ea13-72cc-44cf-88b2-8fb8541a409b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:34.521918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8832ea13-72cc-44cf-88b2-8fb8541a409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:34.537215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:34.552938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aa2f7-04dd-4ad2-97bb-cf9e81ae2a9e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:34.593860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:34.594465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.594740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.595030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.595673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:34.595972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:34.597346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:34.603558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 423/856] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:34.609937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:34.616053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 434/857] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:34.639962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832ea13-72cc-44cf-88b2-8fb8541a409b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:34.668707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:34.736614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca697e2-f6a9-4a7a-b196-f6738506e7ef None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:34.757055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832ea13-72cc-44cf-88b2-8fb8541a409b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:34.783112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:34.784558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.785039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.785490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.786708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:34.787220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:34.789136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:34.794859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a654e-53de-4459-8f29-bbaed77498c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:34.795659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a654e-53de-4459-8f29-bbaed77498c3 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.795899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a654e-53de-4459-8f29-bbaed77498c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.796001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 424/858] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:34.796258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a654e-53de-4459-8f29-bbaed77498c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.813956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a654e-53de-4459-8f29-bbaed77498c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-be6a654e-53de-4459-8f29-bbaed77498c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:34.818945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:34.820033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:34.821759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 435/859] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:04:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:34.829093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:34.944724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c43674-ec76-4a32-a2f6-551cc2c257bd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:34.957185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:34.986304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:34.987453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:34.988088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:34.988830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:34.989812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:34.990117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:34.991513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:35.005217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 425/860] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:04:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:35.006641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a654e-53de-4459-8f29-bbaed77498c3 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:35.018807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:35.127048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a654e-53de-4459-8f29-bbaed77498c3 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:35.162319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:35.162941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:35.163250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:35.163517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:35.174844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:35.176331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:35.190847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:35.191913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:35.192831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:35.192831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:35.192831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:35.192831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 436/861] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:35.198750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:35.302855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97da500-cc2e-4349-9462-0d9e24eab1b1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:35.337863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4259b-34a0-47ae-b01a-bcf205b1c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:35.338467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4259b-34a0-47ae-b01a-bcf205b1c88c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:35.338793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4259b-34a0-47ae-b01a-bcf205b1c88c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:35.339092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4259b-34a0-47ae-b01a-bcf205b1c88c None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:35.339733 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4259b-34a0-47ae-b01a-bcf205b1c88c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:35.340083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4259b-34a0-47ae-b01a-bcf205b1c88c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:35.341500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4259b-34a0-47ae-b01a-bcf205b1c88c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:35.359214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 426/862] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:35.364081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:35.427862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:35.471263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3953b5-bbde-438c-a66d-07eb59b611a6 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:35.507275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:35.507901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:35.508177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:35.508444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:35.516287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:35.517729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:35.527934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6165ea-f58f-435d-8520-a90704e3028e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:35.529174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6165ea-f58f-435d-8520-a90704e3028e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:35.529174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6165ea-f58f-435d-8520-a90704e3028e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:35.529737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:35.530687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:35.531689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:35.532235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 427/863] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:35.532313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 437/864] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:04:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:35.535032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50d8f-392b-4bff-a585-53625fe1f9c6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:35.535336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50d8f-392b-4bff-a585-53625fe1f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:35.535622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50d8f-392b-4bff-a585-53625fe1f9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:35.536566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:35.554478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-69a50d8f-392b-4bff-a585-53625fe1f9c6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:35.556423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-69a50d8f-392b-4bff-a585-53625fe1f9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:35.587501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:35.635297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 428/865] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:35.640347 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108493c-5c13-4220-b68a-c9218090f16a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:35.660031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50d8f-392b-4bff-a585-53625fe1f9c6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:35.660850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:35.678117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:35.678715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:35.678990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:35.679280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:35.680006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:35.680303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:35.682264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:35.704644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 438/866] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 12:04:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:35.712527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:35.767908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50d8f-392b-4bff-a585-53625fe1f9c6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:35.804173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb140858-94df-417c-8e64-88578b00c7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:35.804839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140858-94df-417c-8e64-88578b00c7ed None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:35.805061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140858-94df-417c-8e64-88578b00c7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:35.805329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140858-94df-417c-8e64-88578b00c7ed None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:35.806520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb140858-94df-417c-8e64-88578b00c7ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:35.807928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb140858-94df-417c-8e64-88578b00c7ed None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:35.832244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18522a7b-d801-45e2-a772-1e1ddb73cdc7 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:35.868890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c759d-aa85-4143-89f7-a5132e118099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:35.869898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c759d-aa85-4143-89f7-a5132e118099 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:35.870388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c759d-aa85-4143-89f7-a5132e118099 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:35.870834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c759d-aa85-4143-89f7-a5132e118099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:35.883043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 429/867] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:04:35 2022] POST /identity/v3/services => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:35.928542 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:35.957646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c759d-aa85-4143-89f7-a5132e118099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:35.959175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c759d-aa85-4143-89f7-a5132e118099 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:36.078221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 439/868] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:04:36.083944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb140858-94df-417c-8e64-88578b00c7ed None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:36.109227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:36.219275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb140858-94df-417c-8e64-88578b00c7ed None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:36.255446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455b3a3-9ea8-4a97-aed4-9d228f2fb3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:36.256077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b3a3-9ea8-4a97-aed4-9d228f2fb3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:36.256362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b3a3-9ea8-4a97-aed4-9d228f2fb3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:36.256741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b3a3-9ea8-4a97-aed4-9d228f2fb3d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:36.267938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455b3a3-9ea8-4a97-aed4-9d228f2fb3d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:36.269446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455b3a3-9ea8-4a97-aed4-9d228f2fb3d5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-0455b3a3-9ea8-4a97-aed4-9d228f2fb3d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:36.284386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:36.285342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:36.286265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:36.286265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:36.286265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:36.289116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 430/869] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:36.292234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c759d-aa85-4143-89f7-a5132e118099 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:36.346462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc25e5c-1a60-4c57-9f22-5d9f0f04ea71 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:36.346636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc25e5c-1a60-4c57-9f22-5d9f0f04ea71 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:36.346918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc25e5c-1a60-4c57-9f22-5d9f0f04ea71 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:36.348522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 431/870] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:36.353953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52051a5a-0715-45c1-9e35-e4ae359bffe3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:36.354235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52051a5a-0715-45c1-9e35-e4ae359bffe3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:36.354688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52051a5a-0715-45c1-9e35-e4ae359bffe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:36.375194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-52051a5a-0715-45c1-9e35-e4ae359bffe3 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:36.376388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-52051a5a-0715-45c1-9e35-e4ae359bffe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:36.377483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:36.402099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c759d-aa85-4143-89f7-a5132e118099 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:36.438851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:36.439504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:36.439851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:36.440232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:36.440924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:36.441295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:36.442734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:36.448851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:36.455052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 440/871] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:36.459959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 432/872] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:36.463040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:36.483981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52051a5a-0715-45c1-9e35-e4ae359bffe3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:36.593931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c9f15-68bf-4159-851b-a4530edf6ae9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:36.597483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52051a5a-0715-45c1-9e35-e4ae359bffe3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:36.636508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:36.637139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:36.637519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:36.637904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:36.638519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:36.639812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:36.640079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:36.640983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:36.641451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:36.641929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:36.645996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:36.651315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:36.655911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:36.656762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:36.657963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:36.657963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:36.658148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 433/873] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:36.659078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:36.673849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:36.675279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:36.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:36.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:36.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:36.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:36.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:36.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:36.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:36.676263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:36.683325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 441/874] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:36.685456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:36.740942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:36.798040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262141e-ad20-4930-b027-53ec0164111f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:36.804609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df966d4-3535-4040-a8ca-d7c4db955624 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:36.843944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:36.844549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:36.844852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:36.845128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:36.845817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:36.846122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:36.847645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:36.850487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:36.851341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:36.851824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:36.852911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:36.853835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:36.854362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:36.856566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:36.860853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 442/875] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:36.867078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 434/876] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:36.871031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:36.871291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:36.941057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:36.986963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4518-2baf-4cb2-b246-d30dc21ca771 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:36.991107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbafa9-45be-4fc3-8684-5b50aee529da None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:37.017127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:37.022517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57726aae-0282-43cb-8bfe-e5b2d356ea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:37.027100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726aae-0282-43cb-8bfe-e5b2d356ea03 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:37.027100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726aae-0282-43cb-8bfe-e5b2d356ea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:37.027100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726aae-0282-43cb-8bfe-e5b2d356ea03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:37.027100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57726aae-0282-43cb-8bfe-e5b2d356ea03 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=87a5db9a38544dc189b0c0d7bb205d83) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:37.027100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57726aae-0282-43cb-8bfe-e5b2d356ea03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:37.027100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57726aae-0282-43cb-8bfe-e5b2d356ea03 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:37.027521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cd543-550a-49de-9631-0b6668bc3376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:37.028106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd543-550a-49de-9631-0b6668bc3376 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:37.028411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd543-550a-49de-9631-0b6668bc3376 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:37.028820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd543-550a-49de-9631-0b6668bc3376 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:37.029868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cd543-550a-49de-9631-0b6668bc3376 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:37.030200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cd543-550a-49de-9631-0b6668bc3376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:37.031560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cd543-550a-49de-9631-0b6668bc3376 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:37.032782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 443/877] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:36 2022] GET /identity/v3/role_assignments?user.id=87a5db9a38544dc189b0c0d7bb205d83&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:37.039050 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 435/878] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:37.045653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921a9f0-f85d-4c93-97cf-8a9e2fa3fe77 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:37.045653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921a9f0-f85d-4c93-97cf-8a9e2fa3fe77 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:37.045653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921a9f0-f85d-4c93-97cf-8a9e2fa3fe77 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:37.045653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 444/879] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:37.055097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6b068-8e80-41ca-8388-da11c7450f8a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:37.055097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6b068-8e80-41ca-8388-da11c7450f8a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:37.055097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6b068-8e80-41ca-8388-da11c7450f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:37.074017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-aaf6b068-8e80-41ca-8388-da11c7450f8a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:37.075413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-aaf6b068-8e80-41ca-8388-da11c7450f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:37.164397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 436/880] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:37 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:37.188982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921a9f0-f85d-4c93-97cf-8a9e2fa3fe77 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:37.305108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921a9f0-f85d-4c93-97cf-8a9e2fa3fe77 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:37.328879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:37.342774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba698c-bd05-438e-9b7b-e850059dca8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:37.343407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba698c-bd05-438e-9b7b-e850059dca8a None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:37.343690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba698c-bd05-438e-9b7b-e850059dca8a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:37.343962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba698c-bd05-438e-9b7b-e850059dca8a None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:37.344618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba698c-bd05-438e-9b7b-e850059dca8a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:37.346062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba698c-bd05-438e-9b7b-e850059dca8a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:37.451409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 445/881] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:04:37 2022] POST /identity/v3/services => generated 269 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:38.054599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e534794-9075-465b-8095-29966d25b841 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:38.055202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e534794-9075-465b-8095-29966d25b841 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:38.055202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e534794-9075-465b-8095-29966d25b841 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:38.057354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 437/882] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:38.062536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568014e-4320-4366-b139-4ca8a0e67e0d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:38.063102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568014e-4320-4366-b139-4ca8a0e67e0d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:38.063578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568014e-4320-4366-b139-4ca8a0e67e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:38.082221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d568014e-4320-4366-b139-4ca8a0e67e0d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:38.083853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d568014e-4320-4366-b139-4ca8a0e67e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:38.171564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 446/883] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:38.193559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e534794-9075-465b-8095-29966d25b841 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:38.304819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e534794-9075-465b-8095-29966d25b841 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:38.328155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:38.343691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:38.349203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:38.349203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:38.349203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:38.349203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:38.349203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:38.356566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:38.357550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:38.359142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 438/884] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:38.363527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568014e-4320-4366-b139-4ca8a0e67e0d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:38.470913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568014e-4320-4366-b139-4ca8a0e67e0d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:38.479114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:38.506771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:38.507416 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:38.507695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:38.507974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:38.508629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:38.508933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:38.510309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:38.515284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 447/885] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:38.520482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:38.634202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545730c-889c-44b5-b913-38c214af77ea None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:38.670345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:38.670955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:38.671263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:38.671538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:38.672172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:38.672476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:38.673830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:38.679235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 439/886] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:38.688262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:38.791702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ee02c-6bbb-43ea-9138-5f120de6a2a9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:38.831155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7948fb-fe19-4f1f-a6ad-89bce8479eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:38.831763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7948fb-fe19-4f1f-a6ad-89bce8479eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:38.832046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7948fb-fe19-4f1f-a6ad-89bce8479eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:38.832318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7948fb-fe19-4f1f-a6ad-89bce8479eb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:38.833020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7948fb-fe19-4f1f-a6ad-89bce8479eb3 None admin] RBAC: Adding query filter params (interface=public, service_id=90ef155b27be49bdaea5fa850e8684c8, region_id=RegionOne) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:38.833332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7948fb-fe19-4f1f-a6ad-89bce8479eb3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:38.834717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7948fb-fe19-4f1f-a6ad-89bce8479eb3 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:38.840831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 448/887] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 12:04:38 2022] GET /identity/v3/endpoints?service_id=90ef155b27be49bdaea5fa850e8684c8&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:38.848145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:38.898525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:38.951815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f715eb-d913-449b-84b6-39f5dcfd06b5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:38.989829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb6985-ec0f-4d5c-89ac-e3da1538b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:38.990436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb6985-ec0f-4d5c-89ac-e3da1538b930 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:38.990715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb6985-ec0f-4d5c-89ac-e3da1538b930 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:38.990975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb6985-ec0f-4d5c-89ac-e3da1538b930 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:38.991613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb6985-ec0f-4d5c-89ac-e3da1538b930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:38.993039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb6985-ec0f-4d5c-89ac-e3da1538b930 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:38.999482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 440/888] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:04:38 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:04:39.262761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954b7ee-ae60-4b95-9aa5-fda4ea43f605 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.262761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954b7ee-ae60-4b95-9aa5-fda4ea43f605 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.262761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954b7ee-ae60-4b95-9aa5-fda4ea43f605 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.263754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 449/889] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:39.266236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26a47-ff4b-4976-b5c4-69bedf1d0423 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.266764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26a47-ff4b-4976-b5c4-69bedf1d0423 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.266764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26a47-ff4b-4976-b5c4-69bedf1d0423 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.268384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 441/890] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:39.272017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785a4b-569d-44ce-82e0-e562a26e81cb None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.272017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785a4b-569d-44ce-82e0-e562a26e81cb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.272017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785a4b-569d-44ce-82e0-e562a26e81cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.273340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413245b-1a8c-4767-bf1e-74e44c1eacf0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.273621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413245b-1a8c-4767-bf1e-74e44c1eacf0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.273898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413245b-1a8c-4767-bf1e-74e44c1eacf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.291370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-16785a4b-569d-44ce-82e0-e562a26e81cb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:39.292608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4413245b-1a8c-4767-bf1e-74e44c1eacf0 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:39.293143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-16785a4b-569d-44ce-82e0-e562a26e81cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:39.293915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4413245b-1a8c-4767-bf1e-74e44c1eacf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:39.327389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:39.368717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 442/892] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:39.369102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 450/892] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:39.388411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413245b-1a8c-4767-bf1e-74e44c1eacf0 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:39.389993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785a4b-569d-44ce-82e0-e562a26e81cb None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:39.435467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:39.494065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413245b-1a8c-4767-bf1e-74e44c1eacf0 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:39.495998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785a4b-569d-44ce-82e0-e562a26e81cb None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:39.530823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800dfe6-7b93-4333-8561-fafda4714b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:39.531367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800dfe6-7b93-4333-8561-fafda4714b77 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.531649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800dfe6-7b93-4333-8561-fafda4714b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.531954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800dfe6-7b93-4333-8561-fafda4714b77 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.532584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800dfe6-7b93-4333-8561-fafda4714b77 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:39.533522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e25c2-c066-4e65-bfb8-fad8c0a74551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:39.533953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800dfe6-7b93-4333-8561-fafda4714b77 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:39.534656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e25c2-c066-4e65-bfb8-fad8c0a74551 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.535182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e25c2-c066-4e65-bfb8-fad8c0a74551 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.535653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e25c2-c066-4e65-bfb8-fad8c0a74551 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.546674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e25c2-c066-4e65-bfb8-fad8c0a74551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-485e25c2-c066-4e65-bfb8-fad8c0a74551 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:39.551572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:39.552572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:04:39.553464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:39.555074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 451/893] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 12:04:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:39.559569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e25c2-c066-4e65-bfb8-fad8c0a74551 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:39.586114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:39.625650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 443/894] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:04:39 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:39.627498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de37b5-2cac-4960-a7ab-89fa8a7872e9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.627996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de37b5-2cac-4960-a7ab-89fa8a7872e9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.628481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de37b5-2cac-4960-a7ab-89fa8a7872e9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.629966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 444/895] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:39.637425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2a32-a4be-40ab-a611-0ee3fbe89dc8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.637717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2a32-a4be-40ab-a611-0ee3fbe89dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.637975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2a32-a4be-40ab-a611-0ee3fbe89dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.679529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-293b2a32-a4be-40ab-a611-0ee3fbe89dc8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:39.682027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-293b2a32-a4be-40ab-a611-0ee3fbe89dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:39.684922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e25c2-c066-4e65-bfb8-fad8c0a74551 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:39.732675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:39.733654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.734142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.734619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.735583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:39.736101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:39.737946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:39.750607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 452/896] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 12:04:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:39.756343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:39.775109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 445/897] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:39.798693 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b2a32-a4be-40ab-a611-0ee3fbe89dc8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:39.831386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:39.866112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:39.895986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84f936-c015-46ef-95dd-080169ed11fc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:39.922088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b2a32-a4be-40ab-a611-0ee3fbe89dc8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:39.934746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:39.935806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.936308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.936805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.952120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:39.954078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:39.963245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:39.963889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:39.964159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:39.964435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:39.965423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:39.967127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:39.972142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:39.973103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:39.974011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:39.974011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:39.974011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:39.980492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:39.981448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:39.982744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 446/899] 10.222.0.30 () {56 vars in 1132 bytes} [Sun Aug 7 12:04:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:39.983591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 453/899] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:39.983673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:39.987448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:40.102276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ff233-cd70-4f0f-8aa1-2511ea18c10b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:40.107501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:40.116782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98182f77-201c-4a94-badf-747b25981c07 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:40.150115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:40.151298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:40.151682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:40.152498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:40.153426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:40.153970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:40.155691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:40.158274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:40.166891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:40.173878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:40.173878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:40.173878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:40.173878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:40.173878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:40.173878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:40.174370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 454/900] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:40.176904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:40.183838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 447/901] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:40.189269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:40.301476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df8983-6a8e-40a0-9b30-42696a3e54b6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:40.318545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb32fc-55c0-4988-b5ac-0e8b710f05fe None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:40.341247 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:40.342615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:40.343337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:40.343790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:40.344820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:40.345275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:40.347512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:40.355390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 448/902] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:40.361238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:40.362205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:40.362835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:40.363152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:40.363424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:40.373615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:40.375613 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:40.391296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:40.393452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:40.394404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:40.395415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 455/903] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:04:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:40.409690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:40.474572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa9b02-b7b8-4802-a907-5de446b10676 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:40.480164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:40.513403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:40.514432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:40.514936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:40.515471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:40.516361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] RBAC: Adding query filter params (interface=public, service_id=96628e79c69a484a814e3a00751c6722, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:40.516893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:40.518576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:40.523859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab6dc7-f182-4f95-995d-837e01b8de25 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:40.529442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 449/904] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 12:04:40 2022] GET /identity/v3/endpoints?service_id=96628e79c69a484a814e3a00751c6722&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:40.530611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:40.563647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:40.564648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:40.564791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:40.565109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:40.565861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:40.566194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:40.567759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:40.588014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 456/905] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 12:04:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:40.593528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:40.628576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:40.641508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a3c86-6800-4759-a748-f92489c393fc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:40.679702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d0a72-96f3-45fd-9112-3603ea2eff99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:40.680711 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d0a72-96f3-45fd-9112-3603ea2eff99 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:40.681173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d0a72-96f3-45fd-9112-3603ea2eff99 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:40.681623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d0a72-96f3-45fd-9112-3603ea2eff99 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:40.682417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d0a72-96f3-45fd-9112-3603ea2eff99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:40.684097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d0a72-96f3-45fd-9112-3603ea2eff99 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:40.695376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 450/906] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:04:40 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:04:40.726298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f04ab-9256-4b21-9f9f-6bba5d29275e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:40.743483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:40.766963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d306bbc-c280-4ea3-9946-61ccfd77926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:40.767612 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306bbc-c280-4ea3-9946-61ccfd77926b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:40.767895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306bbc-c280-4ea3-9946-61ccfd77926b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:40.768735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306bbc-c280-4ea3-9946-61ccfd77926b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:40.847421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d306bbc-c280-4ea3-9946-61ccfd77926b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:40.848896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d306bbc-c280-4ea3-9946-61ccfd77926b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:40.957261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 457/907] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:40 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:04:40.958677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d0a72-96f3-45fd-9112-3603ea2eff99 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.066616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d0a72-96f3-45fd-9112-3603ea2eff99 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:41.091902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7232909-bcf3-45da-91e9-eeb6e769529f None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.092217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7232909-bcf3-45da-91e9-eeb6e769529f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.092666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7232909-bcf3-45da-91e9-eeb6e769529f None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.098593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 458/908] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:41.100770 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbbd3f-515d-49c1-ab3f-3d2e62071d3f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.101063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbbd3f-515d-49c1-ab3f-3d2e62071d3f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.101339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbbd3f-515d-49c1-ab3f-3d2e62071d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.112578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:41.113221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.113503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.113779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.125519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:41.126071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c0fbbd3f-515d-49c1-ab3f-3d2e62071d3f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:41.126958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:41.127305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c0fbbd3f-515d-49c1-ab3f-3d2e62071d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:41.143059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:41.144067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:41.144967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:41.144967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:41.144967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:41.144967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 451/909] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:41.150936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.151329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:41.155811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:41.223100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 459/910] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:41.238741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbbd3f-515d-49c1-ab3f-3d2e62071d3f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.281041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f517c6-45fb-4615-9c92-394e715e7af1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:41.318797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:41.319453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.319675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.319947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.320585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:41.320885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:41.322863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:41.340523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 452/911] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:41.342919 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.357056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbbd3f-515d-49c1-ab3f-3d2e62071d3f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:41.401697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:41.402765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.403073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.403349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.404193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:41.405599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:41.418023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:41.419149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 12:04:41.420099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:41.423354 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 460/912] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:41.426324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.436249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:41.440628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:41.485423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2f1a2-e6de-4798-a71a-77311333cf31 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:41.531759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:41.532471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.532803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.533080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.542348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:41.543866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:41.557987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:41.558902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:41.566899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 453/913] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:41.567446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6b63a-8292-43af-b398-93bc6dcce685 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:41.567874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.605463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:41.606163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.606475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.606751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.607678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:41.607861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:41.609177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:41.619045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 461/914] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:41.622538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.692115 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99554c41-e192-4485-a52c-0623921925f1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:41.698356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:41.717977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:41.729973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e700390-7266-4e95-912a-263c8b0b107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:41.730640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700390-7266-4e95-912a-263c8b0b107e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.730923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700390-7266-4e95-912a-263c8b0b107e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.731224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700390-7266-4e95-912a-263c8b0b107e None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.731463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dad02-4982-447d-ae08-fc74f17e4204 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:41.731963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e700390-7266-4e95-912a-263c8b0b107e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:41.732228 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e700390-7266-4e95-912a-263c8b0b107e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:41.733702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e700390-7266-4e95-912a-263c8b0b107e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:41.751613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 454/915] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:41.751613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffc49-7d30-4ab7-af82-61083d217a1d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.751613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffc49-7d30-4ab7-af82-61083d217a1d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.751613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffc49-7d30-4ab7-af82-61083d217a1d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.751613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 455/916] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:41.755235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfffc49-7d30-4ab7-af82-61083d217a1d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.774901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf8be4-9c5d-43c1-bc9e-b0e2301b30ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:41.775618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf8be4-9c5d-43c1-bc9e-b0e2301b30ba None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.775907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf8be4-9c5d-43c1-bc9e-b0e2301b30ba None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.776183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf8be4-9c5d-43c1-bc9e-b0e2301b30ba None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.776815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf8be4-9c5d-43c1-bc9e-b0e2301b30ba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:41.777157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf8be4-9c5d-43c1-bc9e-b0e2301b30ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:41.778969 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf8be4-9c5d-43c1-bc9e-b0e2301b30ba None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:41.788258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 462/917] 10.222.0.30 () {56 vars in 1114 bytes} [Sun Aug 7 12:04:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:41.788258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c6bcb-dd13-42c2-b4c1-f3d3e5187ca6 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.788258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c6bcb-dd13-42c2-b4c1-f3d3e5187ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.788258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c6bcb-dd13-42c2-b4c1-f3d3e5187ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.804743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a16c6bcb-dd13-42c2-b4c1-f3d3e5187ca6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:41.806050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a16c6bcb-dd13-42c2-b4c1-f3d3e5187ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:41.872445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfffc49-7d30-4ab7-af82-61083d217a1d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:41.879108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 463/918] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:41.880677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c6bcb-dd13-42c2-b4c1-f3d3e5187ca6 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.912622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:41.913231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:41.913530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:41.913834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:41.915396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] RBAC: Adding query filter params (role.id=93f3167474ff4071a8c82c01f6fd3f1c, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=87a5db9a38544dc189b0c0d7bb205d83) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:41.915510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:41.917052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:41.931034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 456/919] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:41 2022] GET /identity/v3/role_assignments?user.id=87a5db9a38544dc189b0c0d7bb205d83&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=93f3167474ff4071a8c82c01f6fd3f1c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:41.931034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:41.974745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:41.998368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:42.024156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c6bcb-dd13-42c2-b4c1-f3d3e5187ca6 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:42.068026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59efa8-c4a8-4108-9479-49d89ea0b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:42.069712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59efa8-c4a8-4108-9479-49d89ea0b602 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:42.070258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59efa8-c4a8-4108-9479-49d89ea0b602 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:42.070738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59efa8-c4a8-4108-9479-49d89ea0b602 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:42.071828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59efa8-c4a8-4108-9479-49d89ea0b602 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:42.071948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe349-e82e-4de5-98c8-ab948e6e079f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:42.073879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59efa8-c4a8-4108-9479-49d89ea0b602 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:42.113077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:42.113340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:42.113549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:42.119023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:42.119023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:42.119023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:42.127371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:42.128377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:42.129244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 457/920] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:42.134960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:42.168564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 464/921] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:04:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:42.260326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451fe7-bdce-46d0-be21-d2561faef148 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:42.277504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:42.307506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:42.308125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:42.308404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:42.308678 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:42.309313 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:42.309615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:42.310990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:42.316867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 458/922] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:42.322835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59efa8-c4a8-4108-9479-49d89ea0b602 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:42.436862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59efa8-c4a8-4108-9479-49d89ea0b602 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:42.461463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:42.477894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:42.478912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:42.479408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:42.479896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:42.480660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:42.480968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:42.482344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:42.487801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 465/923] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:42.494570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:42.600229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a96db5-32eb-493a-b806-f5c1c85f0510 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:42.640384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dbfa5-23f6-4166-b382-65a21f3c6655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:42.640987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dbfa5-23f6-4166-b382-65a21f3c6655 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:42.641255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dbfa5-23f6-4166-b382-65a21f3c6655 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:42.641513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dbfa5-23f6-4166-b382-65a21f3c6655 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:42.642211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dbfa5-23f6-4166-b382-65a21f3c6655 None admin] RBAC: Adding query filter params (interface=public, service_id=623ec9e122ce4678b4b02d49099cfc78, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:42.642513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dbfa5-23f6-4166-b382-65a21f3c6655 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:42.643894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dbfa5-23f6-4166-b382-65a21f3c6655 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:42.649890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 459/924] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 12:04:42 2022] GET /identity/v3/endpoints?service_id=623ec9e122ce4678b4b02d49099cfc78&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:42.655215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:42.762657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3fd9f-9d8e-4dab-9174-a8b28e8ba920 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:42.786593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9a2aa-ccc3-4e4f-b44d-de5dfa94e0ea None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:42.786909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9a2aa-ccc3-4e4f-b44d-de5dfa94e0ea None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:42.787426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9a2aa-ccc3-4e4f-b44d-de5dfa94e0ea None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:42.788761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 460/925] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:42.794620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1213ac-42ea-4daf-ae5c-b8ba2d041f37 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:42.794898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1213ac-42ea-4daf-ae5c-b8ba2d041f37 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:42.795203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1213ac-42ea-4daf-ae5c-b8ba2d041f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:42.801621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb905ba8-9807-4171-b004-6010a8877ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:42.802495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb905ba8-9807-4171-b004-6010a8877ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:42.802980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb905ba8-9807-4171-b004-6010a8877ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:42.803486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb905ba8-9807-4171-b004-6010a8877ae9 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:42.804297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb905ba8-9807-4171-b004-6010a8877ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:42.806326 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb905ba8-9807-4171-b004-6010a8877ae9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:42.813077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 466/926] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:04:42 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:04:42.814279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5c1213ac-42ea-4daf-ae5c-b8ba2d041f37 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:42.823034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5c1213ac-42ea-4daf-ae5c-b8ba2d041f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:42.851490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:42.899190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 461/927] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:42.916268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb905ba8-9807-4171-b004-6010a8877ae9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:42.974411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:43.038478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb905ba8-9807-4171-b004-6010a8877ae9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:43.077226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:43.078295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:43.078803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:43.079318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:43.080132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:43.081759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:43.092496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:43.093548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 12:04:43.094465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:43.102095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 467/928] 10.222.0.30 () {56 vars in 1132 bytes} [Sun Aug 7 12:04:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:43.103817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1213ac-42ea-4daf-ae5c-b8ba2d041f37 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:43.207170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1213ac-42ea-4daf-ae5c-b8ba2d041f37 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:43.242112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:43.243238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:43.243619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:43.243892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:43.244518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:43.244814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:43.246157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:43.254756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 462/929] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:43.256937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:43.367547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7269218-5dd3-4560-80f2-2aeb945c45eb None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:43.382271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:43.404332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70967989-da15-4683-96ed-ae752563d55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:43.404940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70967989-da15-4683-96ed-ae752563d55f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:43.405244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70967989-da15-4683-96ed-ae752563d55f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:43.405513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70967989-da15-4683-96ed-ae752563d55f None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:43.406143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70967989-da15-4683-96ed-ae752563d55f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:43.406443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70967989-da15-4683-96ed-ae752563d55f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:43.407834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70967989-da15-4683-96ed-ae752563d55f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:43.418278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 468/930] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:43.419674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:43.458725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:43.528757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cede12-e211-4469-b880-2fc2974fb488 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:43.564931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7119b-06ee-40f0-b699-96b37b6b1313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:43.565542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7119b-06ee-40f0-b699-96b37b6b1313 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:43.565821 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7119b-06ee-40f0-b699-96b37b6b1313 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:43.566095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7119b-06ee-40f0-b699-96b37b6b1313 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:43.566762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7119b-06ee-40f0-b699-96b37b6b1313 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:43.568194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7119b-06ee-40f0-b699-96b37b6b1313 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:43.645615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 463/931] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:04:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:44.026461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddb057-fdb0-4248-abdc-820c047ef4f5 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.026960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddb057-fdb0-4248-abdc-820c047ef4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.027240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddb057-fdb0-4248-abdc-820c047ef4f5 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.028972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 469/932] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:44.034624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b95292-6248-4c06-8287-e7085c17c1f1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.035090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b95292-6248-4c06-8287-e7085c17c1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.035485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b95292-6248-4c06-8287-e7085c17c1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.056395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-12b95292-6248-4c06-8287-e7085c17c1f1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:44.057923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-12b95292-6248-4c06-8287-e7085c17c1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:44.135410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 464/933] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:44.157210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddb057-fdb0-4248-abdc-820c047ef4f5 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:44.267315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddb057-fdb0-4248-abdc-820c047ef4f5 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:44.302809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5184f6-94bb-44fc-91e6-5837fb98addf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:44.303385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5184f6-94bb-44fc-91e6-5837fb98addf None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.303708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5184f6-94bb-44fc-91e6-5837fb98addf None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.303985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5184f6-94bb-44fc-91e6-5837fb98addf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.316053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5184f6-94bb-44fc-91e6-5837fb98addf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6f5184f6-94bb-44fc-91e6-5837fb98addf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:44.320762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:44.321728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:04:44.322603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:44.323285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 470/934] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 12:04:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:44.329877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b95292-6248-4c06-8287-e7085c17c1f1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:44.351407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:44.367143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1bbd-5924-4b97-bfc7-8aeeecc53752 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.367439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1bbd-5924-4b97-bfc7-8aeeecc53752 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.367679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1bbd-5924-4b97-bfc7-8aeeecc53752 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.369361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 471/935] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:44.376661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc645f-6401-4b2a-b183-57ab00ed7199 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.377032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc645f-6401-4b2a-b183-57ab00ed7199 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.385815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc645f-6401-4b2a-b183-57ab00ed7199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.396112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0bbc645f-6401-4b2a-b183-57ab00ed7199 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:44.397369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0bbc645f-6401-4b2a-b183-57ab00ed7199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:44.430277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:44.435670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b95292-6248-4c06-8287-e7085c17c1f1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:44.472447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:44.473053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.473330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.473601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.474270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:44.474574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:44.475974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:44.477806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 472/936] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:44.485644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 465/937] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 12:04:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:44.491330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc645f-6401-4b2a-b183-57ab00ed7199 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:44.497833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:44.602029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc645f-6401-4b2a-b183-57ab00ed7199 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:44.619174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b94ae6-9220-433f-867b-0ba5b46e9e2e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:44.627812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:44.646209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:44.647267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.647620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.647966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.661358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:44.663031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:44.663173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:44.663723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.664077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.664354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.665053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:44.666985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:44.669598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING keystone.server.flask.common [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:44.680062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:44.682687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:44.683785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:44.683785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:44.683785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:44.683785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 473/938] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:44.688412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:44.708406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:44.724329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.sql.core [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 12:04:44.724329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 07 12:04:44.724329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [parameters: {'id': '7cbda720e9244f7a8913d72c542e38bf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 07 12:04:44.724329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80433) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7cbda720e9244f7a8913d72c542e38bf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7cbda720e9244f7a8913d72c542e38bf. Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application conn.query(q) Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 12:04:44.746651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result.read() Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 07 12:04:44.748272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application next(self.gen) Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application yield resource Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application next(self.gen) Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise self.value Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application session.commit() Aug 07 12:04:44.749847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.session.flush() Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application compat.raise_( Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 12:04:44.750992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 12:04:44.751986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application conn.query(q) Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result.read() Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 12:04:44.752798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [parameters: {'id': '7cbda720e9244f7a8913d72c542e38bf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:44.753708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:44.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7cbda720e9244f7a8913d72c542e38bf. Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:44.755536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 466/939] 10.222.0.30 () {60 vars in 1140 bytes} [Sun Aug 7 12:04:44 2022] POST /identity/v3/projects => generated 262 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:44.762953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:44.800705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15588a-a3fb-48ed-a6bf-4354e5eeb6c0 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:44.839607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:44.840239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.840520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.840793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.841490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:44.841798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:44.843202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:44.860801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 474/940] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:44.861636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:44.873914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0178c5-652e-4bcf-b5a6-d14ac520c2a7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:44.906591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:44.913426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:44.921204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:44.921204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:44.921204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:44.922881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:44.924686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:44.937060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:44.937999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:44.938941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:44.938941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:44.938941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:44.938941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:44.938941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:44.938941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:44.938941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:44.938941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:44.943044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 467/941] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:44.944402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:44.945525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:44.969823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8ec99-19b0-4944-b74b-0edef0592b84 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.010424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.011315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.011697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.012183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.020110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.021819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:45.034127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:45.036616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:45.038876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:45.043034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 475/942] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:04:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:45.044430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.053512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01eb7b-3295-4509-b00e-4d5eb8052b31 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.092062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6baaa1-d580-4fc9-9dbf-566594525817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.092832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6baaa1-d580-4fc9-9dbf-566594525817 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.093122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6baaa1-d580-4fc9-9dbf-566594525817 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.093398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6baaa1-d580-4fc9-9dbf-566594525817 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.094068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6baaa1-d580-4fc9-9dbf-566594525817 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:45.094381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6baaa1-d580-4fc9-9dbf-566594525817 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.095837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6baaa1-d580-4fc9-9dbf-566594525817 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.121594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 468/943] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:45.121594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea62cf1-9c2f-4a84-b7ec-a5ac43af0b90 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.121594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea62cf1-9c2f-4a84-b7ec-a5ac43af0b90 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.121594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea62cf1-9c2f-4a84-b7ec-a5ac43af0b90 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.121594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 469/944] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:45.131040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62104b-e466-4bb7-9322-ecec67298da6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.131040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62104b-e466-4bb7-9322-ecec67298da6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.131040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62104b-e466-4bb7-9322-ecec67298da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.146948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ec62104b-e466-4bb7-9322-ecec67298da6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:45.148672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ec62104b-e466-4bb7-9322-ecec67298da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:45.172575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace75f1-be2d-479d-84b6-bb0484f95adf None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.210333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:45.219763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.225252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.225252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.225252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.225252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:45.225252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.225878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.247443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 470/945] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:45.247651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:45.252044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 476/946] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 12:04:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:45.257787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62104b-e466-4bb7-9322-ecec67298da6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.268724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.393371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e1467-1151-438a-a266-76fa5f8cdd08 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.398144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62104b-e466-4bb7-9322-ecec67298da6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.437199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.438214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.438782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.439286 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.440132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.441300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.441849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.442382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.442907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.443353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:45.452087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:45.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:45.454042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:45.454906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.456555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.456935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 477/947] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:45.463292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:45.472762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:45.473881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:45.474793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:45.474793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:45.474793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:45.476783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 471/948] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:45.482573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.516208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:45.529512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:45.578436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad157-e729-4489-8a03-48ceb5978a7d None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.589085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6dfa7-e7b5-4529-af07-29b5a653118a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.621597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.622307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.622663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.623036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.623761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:45.624314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.625589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.632984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.634829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.635446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.635914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.636477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 478/949] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:45.637004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:45.637485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.637669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.639315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.659362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 472/950] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:45.660716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.765482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:45.767895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db265ba8-808c-4704-abdb-31727ce55bb1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.774697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccba01-08a7-41ac-b5a0-24479553c301 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.793546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:45.807690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec050d-d25d-4ac0-9bba-a5417c7bf447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.808297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec050d-d25d-4ac0-9bba-a5417c7bf447 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.808578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec050d-d25d-4ac0-9bba-a5417c7bf447 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.808854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec050d-d25d-4ac0-9bba-a5417c7bf447 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.809493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec050d-d25d-4ac0-9bba-a5417c7bf447 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:45.809796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec050d-d25d-4ac0-9bba-a5417c7bf447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.811205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec050d-d25d-4ac0-9bba-a5417c7bf447 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.811634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc12f-8cef-4cb5-b246-9791a0414b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.812267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc12f-8cef-4cb5-b246-9791a0414b7a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.812552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc12f-8cef-4cb5-b246-9791a0414b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.812831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc12f-8cef-4cb5-b246-9791a0414b7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.821950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 479/951] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:45.821950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47589422-f678-48aa-bcc6-d19281fdaebe None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.821950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47589422-f678-48aa-bcc6-d19281fdaebe None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.821950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47589422-f678-48aa-bcc6-d19281fdaebe None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.822486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fc12f-8cef-4cb5-b246-9791a0414b7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:45.822574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 480/952] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:45.823923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47589422-f678-48aa-bcc6-d19281fdaebe None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.826113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fc12f-8cef-4cb5-b246-9791a0414b7a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-8f6fc12f-8cef-4cb5-b246-9791a0414b7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:45.838210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:45.839230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:45.840725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 473/953] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:45.841228 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa2ec7-5ab7-4433-987c-f3cc0312f5b7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.841228 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa2ec7-5ab7-4433-987c-f3cc0312f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:45.841355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa2ec7-5ab7-4433-987c-f3cc0312f5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:45.859543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-22fa2ec7-5ab7-4433-987c-f3cc0312f5b7 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:45.860743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-22fa2ec7-5ab7-4433-987c-f3cc0312f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:45.932081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 474/954] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:45.932846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa2ec7-5ab7-4433-987c-f3cc0312f5b7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:45.953428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47589422-f678-48aa-bcc6-d19281fdaebe None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:45.998308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc2bea-32ec-4310-8760-7c1cb8bf3ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:45.999310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc2bea-32ec-4310-8760-7c1cb8bf3ccc None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:45.999900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc2bea-32ec-4310-8760-7c1cb8bf3ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:46.000184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc2bea-32ec-4310-8760-7c1cb8bf3ccc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:46.000847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc2bea-32ec-4310-8760-7c1cb8bf3ccc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:46.002337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc2bea-32ec-4310-8760-7c1cb8bf3ccc None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:46.021846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:46.039479 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa2ec7-5ab7-4433-987c-f3cc0312f5b7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:46.073180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:46.083081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:46.083298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 481/955] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:04:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:46.083725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:46.083725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:46.083914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc2bea-32ec-4310-8760-7c1cb8bf3ccc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:46.083998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:46.083998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:46.083998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:46.099872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 475/956] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:46.106625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:46.152972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:46.238148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc2bea-32ec-4310-8760-7c1cb8bf3ccc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:46.261138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caaca3-944e-4e3c-b46a-a92ba7184e27 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:46.277972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:46.278604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:46.278883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:46.279195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:46.279841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:46.281297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:46.291810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:46.292992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:46.293880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 482/957] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:46.299671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:46.310646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f602a-58c1-4799-933c-07312b4520a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:46.311661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f602a-58c1-4799-933c-07312b4520a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:46.312146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f602a-58c1-4799-933c-07312b4520a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:46.312621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f602a-58c1-4799-933c-07312b4520a0 None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/87a5db9a38544dc189b0c0d7bb205d83/roles/93f3167474ff4071a8c82c01f6fd3f1c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:46.315601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f602a-58c1-4799-933c-07312b4520a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=87a5db9a38544dc189b0c0d7bb205d83, role_id=93f3167474ff4071a8c82c01f6fd3f1c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:46.317426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f602a-58c1-4799-933c-07312b4520a0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:46.402214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 476/958] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:04:46 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/87a5db9a38544dc189b0c0d7bb205d83/roles/93f3167474ff4071a8c82c01f6fd3f1c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:04:46.433235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff2343-72aa-4015-9fa7-53b012f02ac9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:46.436343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:46.474829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f10d6-0af6-4ffa-95d6-2801809ac442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:46.475480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f10d6-0af6-4ffa-95d6-2801809ac442 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:46.475758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f10d6-0af6-4ffa-95d6-2801809ac442 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:46.476033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f10d6-0af6-4ffa-95d6-2801809ac442 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:46.476736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f10d6-0af6-4ffa-95d6-2801809ac442 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:46.477700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f10d6-0af6-4ffa-95d6-2801809ac442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:46.479375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f10d6-0af6-4ffa-95d6-2801809ac442 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:46.485536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 483/959] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:46.491911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f602a-58c1-4799-933c-07312b4520a0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:46.495513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:46.606751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f602a-58c1-4799-933c-07312b4520a0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:46.642162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b68579-392c-4bee-94f9-e178bd6feeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:46.642777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68579-392c-4bee-94f9-e178bd6feeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:46.643058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68579-392c-4bee-94f9-e178bd6feeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:46.643332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68579-392c-4bee-94f9-e178bd6feeb9 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:46.643955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68579-392c-4bee-94f9-e178bd6feeb9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:46.644253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68579-392c-4bee-94f9-e178bd6feeb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:46.645724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68579-392c-4bee-94f9-e178bd6feeb9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:46.651471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 477/960] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:47.323373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a823e-b4c4-46d9-90f9-9e5e96d01f86 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:47.324352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a823e-b4c4-46d9-90f9-9e5e96d01f86 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:47.324832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a823e-b4c4-46d9-90f9-9e5e96d01f86 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:47.326607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 484/961] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:47.332243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d84e07-1e09-4459-bf7f-d3668226d578 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:47.332806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d84e07-1e09-4459-bf7f-d3668226d578 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:47.333284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d84e07-1e09-4459-bf7f-d3668226d578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:47.351388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a6d84e07-1e09-4459-bf7f-d3668226d578 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:47.353523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6d84e07-1e09-4459-bf7f-d3668226d578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:47.420567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:47.428374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 478/962] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:47.453091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a823e-b4c4-46d9-90f9-9e5e96d01f86 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:47.565424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a823e-b4c4-46d9-90f9-9e5e96d01f86 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:47.600973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:47.601584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:47.601862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:47.602419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:47.603164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:47.604601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:47.607233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING keystone.server.flask.common [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 07 12:04:47.644395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.sql.core [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 12:04:47.644395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 07 12:04:47.644395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [parameters: {'user_id': '35229edeb9254383bb9800e842b0c37e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 07 12:04:47.644395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80432) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application conn.query(q) Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result.read() Aug 07 12:04:47.652052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 12:04:47.652978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application next(self.gen) Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application yield resource Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application next(self.gen) Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise self.value Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application session.commit() Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 12:04:47.654000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.session.flush() Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application compat.raise_( Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 12:04:47.654940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 12:04:47.655918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application conn.query(q) Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application result.read() Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 12:04:47.656828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [parameters: {'user_id': '35229edeb9254383bb9800e842b0c37e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:47.657702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 07 12:04:47.658531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 07 12:04:47.665549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 07 12:04:47.665549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 07 12:04:47.665549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 12:04:47.665549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 12:04:47.665549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 07 12:04:47.665549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:47.665549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 485/963] 10.222.0.30 () {60 vars in 1131 bytes} [Sun Aug 7 12:04:47 2022] POST /identity/v3/users => generated 160 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:47.668332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d84e07-1e09-4459-bf7f-d3668226d578 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:47.777372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d84e07-1e09-4459-bf7f-d3668226d578 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:47.813005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc00143-4e25-4471-ae5a-f572a656e04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:47.813703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc00143-4e25-4471-ae5a-f572a656e04f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:47.813990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc00143-4e25-4471-ae5a-f572a656e04f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:47.814253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc00143-4e25-4471-ae5a-f572a656e04f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:47.821976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc00143-4e25-4471-ae5a-f572a656e04f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:47.823454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc00143-4e25-4471-ae5a-f572a656e04f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3cc00143-4e25-4471-ae5a-f572a656e04f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:47.835211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:47.836181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:47.837694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 479/964] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:47.845219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:47.878943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:47.954128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177b0bb-c985-4857-97fa-f0a2d5224a19 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:47.994937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42b065-f3b7-466f-b038-42a6d859f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:47.995930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42b065-f3b7-466f-b038-42a6d859f22d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:47.996417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42b065-f3b7-466f-b038-42a6d859f22d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:47.996911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42b065-f3b7-466f-b038-42a6d859f22d None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:47.997820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42b065-f3b7-466f-b038-42a6d859f22d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:47.998327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42b065-f3b7-466f-b038-42a6d859f22d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:48.000198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42b065-f3b7-466f-b038-42a6d859f22d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:48.039803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 486/965] 10.222.0.30 () {56 vars in 1101 bytes} [Sun Aug 7 12:04:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:48.279624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9623-a6a6-4650-987e-e3bb338cdcbe None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.279987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9623-a6a6-4650-987e-e3bb338cdcbe None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.280185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9623-a6a6-4650-987e-e3bb338cdcbe None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.281801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 480/966] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:48.287355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf0be-2172-4b94-9c5d-f29a4fb34cc1 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.287640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf0be-2172-4b94-9c5d-f29a4fb34cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.287917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf0be-2172-4b94-9c5d-f29a4fb34cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.309012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bbcbf0be-2172-4b94-9c5d-f29a4fb34cc1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:48.310469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bbcbf0be-2172-4b94-9c5d-f29a4fb34cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:48.380273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 487/967] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:48.403211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe9623-a6a6-4650-987e-e3bb338cdcbe None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:48.464326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:48.491586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9971d02-326a-4d1c-99a3-cf7f239621d3 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.492177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9971d02-326a-4d1c-99a3-cf7f239621d3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.492459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9971d02-326a-4d1c-99a3-cf7f239621d3 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.494661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 488/968] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:48.499719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db26-2378-4075-8637-68b5eabf4991 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.500077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db26-2378-4075-8637-68b5eabf4991 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.500437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db26-2378-4075-8637-68b5eabf4991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.511574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe9623-a6a6-4650-987e-e3bb338cdcbe None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:48.521737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4794db26-2378-4075-8637-68b5eabf4991 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:48.523299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4794db26-2378-4075-8637-68b5eabf4991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:48.538803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:48.550521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:48.551192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.551474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.551758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.552327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:48.553683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:48.563311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:48.564330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:48.565189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 481/969] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:48.570437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:48.604253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 489/970] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:48.627823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794db26-2378-4075-8637-68b5eabf4991 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:48.679246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76411444-e9c2-4a86-9d19-28f0f3a9b550 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:48.698709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:48.712676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56c75b-8d75-4174-9204-2d736ba74e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:48.713273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56c75b-8d75-4174-9204-2d736ba74e63 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.713548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56c75b-8d75-4174-9204-2d736ba74e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.713826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56c75b-8d75-4174-9204-2d736ba74e63 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.714443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56c75b-8d75-4174-9204-2d736ba74e63 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:48.714739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56c75b-8d75-4174-9204-2d736ba74e63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:48.716127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56c75b-8d75-4174-9204-2d736ba74e63 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:48.720762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 482/971] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:48.721982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbd0a5-18fd-4732-b7bb-0a2026a0f461 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.722270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbd0a5-18fd-4732-b7bb-0a2026a0f461 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.722539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbd0a5-18fd-4732-b7bb-0a2026a0f461 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.723804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 483/972] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:48.727823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbd0a5-18fd-4732-b7bb-0a2026a0f461 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:48.734879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794db26-2378-4075-8637-68b5eabf4991 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:48.768990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7094e7-b3fc-4d54-a612-6b935d58b1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:48.769615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7094e7-b3fc-4d54-a612-6b935d58b1ac None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.769896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7094e7-b3fc-4d54-a612-6b935d58b1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.770169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7094e7-b3fc-4d54-a612-6b935d58b1ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.780663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7094e7-b3fc-4d54-a612-6b935d58b1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-ed7094e7-b3fc-4d54-a612-6b935d58b1ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:48.785313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:48.786440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:48.787396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 490/973] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 12:04:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:48.788130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5738372-7f4f-4565-903c-73715518a3a2 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.788130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5738372-7f4f-4565-903c-73715518a3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.788130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5738372-7f4f-4565-903c-73715518a3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.806105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-b5738372-7f4f-4565-903c-73715518a3a2 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:48.807315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b5738372-7f4f-4565-903c-73715518a3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:48.809489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:48.831866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbd0a5-18fd-4732-b7bb-0a2026a0f461 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:48.864867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:48.865459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:48.865736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:48.866008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:48.866641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:48.866918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:48.868932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:48.873583 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 484/974] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:48.874350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:48.874471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 491/975] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:48.879406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5738372-7f4f-4565-903c-73715518a3a2 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:48.924132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:48.974479 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa0154-3aa5-40c4-8d14-fcc559dd9cd8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:48.980519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5738372-7f4f-4565-903c-73715518a3a2 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:49.008970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.009575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.009859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.010137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:49.010800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:49.011138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:49.012492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:49.013855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.014448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.014733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.015021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:49.015722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] RBAC: Adding query filter params (interface=admin, service_id=623ec9e122ce4678b4b02d49099cfc78, region_id=RegionOne) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:49.016029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:49.017382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:49.020527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 485/976] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 12:04:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:49.021295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:49.027765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 492/977] 10.222.0.30 () {56 vars in 1249 bytes} [Sun Aug 7 12:04:48 2022] GET /identity/v3/endpoints?service_id=623ec9e122ce4678b4b02d49099cfc78&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:49.028564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:49.031545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:49.119726 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d9660-dd7f-430f-bd15-b15f1d8b62e2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:49.128504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7771d-d226-407e-8700-0a2a488fce8f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:49.130114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:49.154539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b8ec3-ead8-4e49-9b1f-aed63e79aff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.155218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b8ec3-ead8-4e49-9b1f-aed63e79aff2 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.155518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b8ec3-ead8-4e49-9b1f-aed63e79aff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.155793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b8ec3-ead8-4e49-9b1f-aed63e79aff2 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:49.156424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b8ec3-ead8-4e49-9b1f-aed63e79aff2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:49.157770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b8ec3-ead8-4e49-9b1f-aed63e79aff2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:49.163528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.164155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.164439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.164739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:49.176011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:49.177446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:49.191714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:49.192836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:49.193930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:49.193930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:49.193930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:49.193930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 493/978] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:49.193930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:49.249096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:49.276932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 486/979] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:04:49 2022] POST /identity/v3/services => generated 242 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:49.277751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b8ec3-ead8-4e49-9b1f-aed63e79aff2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:49.295274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000d026-e0d3-475e-87c6-7d08b0167e8a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:49.343302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f76a3-9258-4789-bed3-d9202e673de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.343917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f76a3-9258-4789-bed3-d9202e673de2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.344194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f76a3-9258-4789-bed3-d9202e673de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.344490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f76a3-9258-4789-bed3-d9202e673de2 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:49.345105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f76a3-9258-4789-bed3-d9202e673de2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:49.346501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f76a3-9258-4789-bed3-d9202e673de2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:49.359595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 494/980] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:04:49 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:04:49.406269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b8ec3-ead8-4e49-9b1f-aed63e79aff2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:49.449309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.450258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.450716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.451203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:49.452005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:49.452490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:49.453870 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:49.469608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 487/981] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:49.474560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f76a3-9258-4789-bed3-d9202e673de2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:49.591534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f76a3-9258-4789-bed3-d9202e673de2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:49.606358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:49.627530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.628442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.628901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.629349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:49.637283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:49.639257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:49.650670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:49.651810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 12:04:49.652834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:49.655083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 495/982] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:04:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:49.658400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:49.690025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:49.768176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d6907-9ffe-4f16-b777-e7decc155849 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:49.804631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.806186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.806675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.807195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:49.808132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:49.808641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:49.810272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:49.829470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 488/983] 10.222.0.30 () {56 vars in 1149 bytes} [Sun Aug 7 12:04:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:49.834676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:49.937846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7e6a7-e2c9-4b1d-b34f-914c2455cf96 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:49.971496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e41820-60d4-4e0d-a410-830cbf2a26a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:49.972097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41820-60d4-4e0d-a410-830cbf2a26a1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:49.972375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41820-60d4-4e0d-a410-830cbf2a26a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:49.972644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41820-60d4-4e0d-a410-830cbf2a26a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:50.016055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:50.042702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e41820-60d4-4e0d-a410-830cbf2a26a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:50.044240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e41820-60d4-4e0d-a410-830cbf2a26a1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:50.150704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 496/984] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:49 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 12:04:50.156158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:50.245015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:50.256663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c040a9-3899-416e-8f3c-82c5a740e7d1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:50.291558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:50.292534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:50.293329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:50.293601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:50.303593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:50.305021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:50.319917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:50.320935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:50.321881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:50.321881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:50.321881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:50.324528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 489/985] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:50.326644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e41820-60d4-4e0d-a410-830cbf2a26a1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:50.402001 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:50.429080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e41820-60d4-4e0d-a410-830cbf2a26a1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:50.465548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:50.466152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:50.466428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:50.466696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:50.467364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:50.467667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:50.469039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:50.483273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 497/986] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:50.486256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:50.592649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfb69c-7ddc-4704-8d70-5ddd369fc181 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:50.628601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:50.629227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:50.629499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:50.629776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:50.636803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:50.638226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:50.640775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:50.650784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:50.653307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:50.654316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 490/987] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:50.657591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:50.757041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047ce24-c12d-4f9c-8ddf-71de553c82cb None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:50.790374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35f3bd-9106-46ee-8a68-6f5cfa1524df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:50.790978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35f3bd-9106-46ee-8a68-6f5cfa1524df None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:50.791283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35f3bd-9106-46ee-8a68-6f5cfa1524df None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:50.791560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35f3bd-9106-46ee-8a68-6f5cfa1524df None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:50.792244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35f3bd-9106-46ee-8a68-6f5cfa1524df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:50.792547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35f3bd-9106-46ee-8a68-6f5cfa1524df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:50.793966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35f3bd-9106-46ee-8a68-6f5cfa1524df None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:50.795548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:50.807785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 498/988] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:50.813466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:50.915885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bf512-36fe-469a-9272-188567bab5fa None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:50.950349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b841da-0bf0-486c-9222-94c4b610521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:50.950941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b841da-0bf0-486c-9222-94c4b610521b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:50.951245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b841da-0bf0-486c-9222-94c4b610521b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:50.951521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b841da-0bf0-486c-9222-94c4b610521b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:50.952252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b841da-0bf0-486c-9222-94c4b610521b None admin] RBAC: Adding query filter params (role.id=93f3167474ff4071a8c82c01f6fd3f1c, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=87a5db9a38544dc189b0c0d7bb205d83) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:50.952562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b841da-0bf0-486c-9222-94c4b610521b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:50.953974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b841da-0bf0-486c-9222-94c4b610521b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:50.967409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 491/989] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:50 2022] GET /identity/v3/role_assignments?user.id=87a5db9a38544dc189b0c0d7bb205d83&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=93f3167474ff4071a8c82c01f6fd3f1c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:51.318293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e793770-7bc2-4f08-8732-19dddd39126e None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.318645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e793770-7bc2-4f08-8732-19dddd39126e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.318821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e793770-7bc2-4f08-8732-19dddd39126e None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.320178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 499/990] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:51.325455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfcd4c-d067-4e88-8328-8e77a958354f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.325743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfcd4c-d067-4e88-8328-8e77a958354f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.326017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfcd4c-d067-4e88-8328-8e77a958354f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.345401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-81cfcd4c-d067-4e88-8328-8e77a958354f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:51.346581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-81cfcd4c-d067-4e88-8328-8e77a958354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:51.418039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:51.427452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 492/991] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:51.428560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676da47-8e25-467e-bd8b-2524f7174596 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.428854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676da47-8e25-467e-bd8b-2524f7174596 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.429132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676da47-8e25-467e-bd8b-2524f7174596 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.431499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 500/992] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:51.436702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc25b4-bfa7-403b-b46c-c32dc3ed234a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.446526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc25b4-bfa7-403b-b46c-c32dc3ed234a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.446526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc25b4-bfa7-403b-b46c-c32dc3ed234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.451238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676da47-8e25-467e-bd8b-2524f7174596 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:51.456612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-41cc25b4-bfa7-403b-b46c-c32dc3ed234a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:51.457874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-41cc25b4-bfa7-403b-b46c-c32dc3ed234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:51.534840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 493/993] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:51.557895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc25b4-bfa7-403b-b46c-c32dc3ed234a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:51.564549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676da47-8e25-467e-bd8b-2524f7174596 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:51.603376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:51.603926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.604207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.604479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.605066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:51.606457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:51.618840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:51.620030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:51.621453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 501/994] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:51.625741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:51.668702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc25b4-bfa7-403b-b46c-c32dc3ed234a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:51.689443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:51.695529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:51.706630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:51.707687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.707992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.708266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.708894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:51.710240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:51.719349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:51.720515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:51.721669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 494/995] 10.222.0.30 () {56 vars in 1126 bytes} [Sun Aug 7 12:04:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:51.726285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:51.741215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d66ed-175c-4334-aff9-361e02cc7921 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:51.780617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:51.781220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.781498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.781767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.782391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] RBAC: Adding query filter params (name=compute) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:51.782696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:51.784155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:51.789129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 502/996] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:51.794941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:51.834107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cca84-d667-43fd-a115-c6408ff65682 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:51.868721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:51.869255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.869533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.869804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.870429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:51.870731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:51.872161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:51.877296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 495/997] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:51.882733 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:51.903633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ffd02-afe2-4559-ba71-874c7a9d36ff None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:51.926784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:51.937465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:51.939495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:51.940097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:51.940379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:51.940654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:51.941284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:51.941585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:51.942960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:51.948579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 503/998] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:51.954202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:51.992069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3583fb1-ce19-4d8f-b327-8c9a252dedb1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:52.026421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:52.027087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:52.027578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:52.027858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:52.028492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:52.028851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:52.030225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:52.035150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 496/999] 10.222.0.30 () {56 vars in 1122 bytes} [Sun Aug 7 12:04:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:52.040818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:52.061151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db89628-8f23-4c32-b9cb-7ddbc9222679 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:52.101449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:52.102165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:52.102498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:52.102865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:52.103619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] RBAC: Adding query filter params (interface=public, service_id=c96f04b91e8d4ac1b8e9879167e37774, region_id=RegionOne) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:52.103936 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:52.105364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:52.122226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 504/1000] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 12:04:51 2022] GET /identity/v3/endpoints?service_id=c96f04b91e8d4ac1b8e9879167e37774&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:52.123104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:52.154555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67ced6-a6c8-4b23-80cb-757f797f74aa None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:52.171253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:52.181639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:52.189700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829ac5e-29b5-45a2-9d31-d6aefa28d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:52.190369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829ac5e-29b5-45a2-9d31-d6aefa28d540 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:52.190702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829ac5e-29b5-45a2-9d31-d6aefa28d540 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:52.190974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829ac5e-29b5-45a2-9d31-d6aefa28d540 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:52.191654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829ac5e-29b5-45a2-9d31-d6aefa28d540 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:52.193003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829ac5e-29b5-45a2-9d31-d6aefa28d540 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:52.231947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca4cbc-c4d6-4fc8-848f-ae24f3c6c846 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:52.263989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd74c5-49cb-4d2e-932c-8a6aa419709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:52.264585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd74c5-49cb-4d2e-932c-8a6aa419709c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:52.264859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd74c5-49cb-4d2e-932c-8a6aa419709c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:52.265128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd74c5-49cb-4d2e-932c-8a6aa419709c None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:52.265795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd74c5-49cb-4d2e-932c-8a6aa419709c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:52.267177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd74c5-49cb-4d2e-932c-8a6aa419709c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:52.273434 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 505/1001] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:04:52 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:04:52.273556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 497/1002] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:04:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:04:53.018401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ee099-5810-42ec-879e-e77f8f89fc2e None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:53.018942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ee099-5810-42ec-879e-e77f8f89fc2e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:53.018942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ee099-5810-42ec-879e-e77f8f89fc2e None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:53.020364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 506/1003] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:53.025906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1112-3b35-4b0e-a7c6-f34b1d0ec184 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:53.026346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1112-3b35-4b0e-a7c6-f34b1d0ec184 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:53.026517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1112-3b35-4b0e-a7c6-f34b1d0ec184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:53.045394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-620c1112-3b35-4b0e-a7c6-f34b1d0ec184 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:53.046583 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-620c1112-3b35-4b0e-a7c6-f34b1d0ec184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:53.127336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 498/1004] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:53.150292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ee099-5810-42ec-879e-e77f8f89fc2e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:53.251416 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ee099-5810-42ec-879e-e77f8f89fc2e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:53.265559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:53.287781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:53.287781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:53.287781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:53.287781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] PATH_INFO: `/v3/services/image` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:53.288159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:53.289210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:53.298456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:53.299694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:04:53.300569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:53.303342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 507/1005] 10.222.0.30 () {56 vars in 1105 bytes} [Sun Aug 7 12:04:53 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:53.310982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c1112-3b35-4b0e-a7c6-f34b1d0ec184 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:53.417120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c1112-3b35-4b0e-a7c6-f34b1d0ec184 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:53.420111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:53.471761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c949b-6983-4e40-a92d-0245287b80f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:53.472784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c949b-6983-4e40-a92d-0245287b80f8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:53.473273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c949b-6983-4e40-a92d-0245287b80f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:53.473766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c949b-6983-4e40-a92d-0245287b80f8 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:53.474639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c949b-6983-4e40-a92d-0245287b80f8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:53.475182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c949b-6983-4e40-a92d-0245287b80f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:53.476768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c949b-6983-4e40-a92d-0245287b80f8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:53.487691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 499/1006] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:53.488946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:53.635514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59435b-a7f0-4c97-ac77-ed98a3f54f77 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:53.671158 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcdb29-d849-42d5-943c-85dac7095491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:53.671762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcdb29-d849-42d5-943c-85dac7095491 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:53.672039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcdb29-d849-42d5-943c-85dac7095491 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:53.672310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcdb29-d849-42d5-943c-85dac7095491 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:53.672939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcdb29-d849-42d5-943c-85dac7095491 None admin] RBAC: Adding query filter params (type=image) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:53.673237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcdb29-d849-42d5-943c-85dac7095491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:53.674675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcdb29-d849-42d5-943c-85dac7095491 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:53.679778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 508/1007] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:54.286804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4f13f-2397-42aa-9160-d36a7aaf00eb None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.287382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4f13f-2397-42aa-9160-d36a7aaf00eb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.287382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4f13f-2397-42aa-9160-d36a7aaf00eb None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.288849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 500/1008] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:54.293832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddd457-dcca-40b8-9a23-9f1098bccdf8 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.294224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddd457-dcca-40b8-9a23-9f1098bccdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.294451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddd457-dcca-40b8-9a23-9f1098bccdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.312534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7dddd457-dcca-40b8-9a23-9f1098bccdf8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:54.313706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7dddd457-dcca-40b8-9a23-9f1098bccdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:54.332820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5c83-76e8-4c1e-9a7b-07dee2cce09f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.333111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5c83-76e8-4c1e-9a7b-07dee2cce09f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.333392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5c83-76e8-4c1e-9a7b-07dee2cce09f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.334878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 501/1009] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:54.340184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec7eed-be0c-4130-86bf-356dfb5bc621 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.340518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec7eed-be0c-4130-86bf-356dfb5bc621 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.340798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec7eed-be0c-4130-86bf-356dfb5bc621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.350302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:54.359361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-baec7eed-be0c-4130-86bf-356dfb5bc621 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:54.360509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-baec7eed-be0c-4130-86bf-356dfb5bc621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:54.383992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 509/1010] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:54.406331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddd457-dcca-40b8-9a23-9f1098bccdf8 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:54.433930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 502/1011] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:54.456839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec7eed-be0c-4130-86bf-356dfb5bc621 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:54.505528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddd457-dcca-40b8-9a23-9f1098bccdf8 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:54.540477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:54.541101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.541379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.541708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.542231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:54.543618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:54.552182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:54.553240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:54.554118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 510/1012] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:54.559018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:54.559473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:54.565393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec7eed-be0c-4130-86bf-356dfb5bc621 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:54.594330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:54.608040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7547-bbf9-4600-a94b-f1d6b1c14af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:54.608669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7547-bbf9-4600-a94b-f1d6b1c14af7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.608943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7547-bbf9-4600-a94b-f1d6b1c14af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.609214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7547-bbf9-4600-a94b-f1d6b1c14af7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.610386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c7547-bbf9-4600-a94b-f1d6b1c14af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:54.611775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c7547-bbf9-4600-a94b-f1d6b1c14af7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:54.613330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 503/1013] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:54.619469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7547-bbf9-4600-a94b-f1d6b1c14af7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:54.668672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7057-c999-45e2-8a27-93421ecd12f8 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:54.705569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:54.706521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.706521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.706810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.707475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] RBAC: Adding query filter params (name=compute) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:54.707776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:54.709362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:54.718911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 511/1014] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:54.719948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:54.730662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7547-bbf9-4600-a94b-f1d6b1c14af7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:54.770172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcbed2-32f9-4dec-aa15-d457bad1a171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:54.770786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcbed2-32f9-4dec-aa15-d457bad1a171 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.771096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcbed2-32f9-4dec-aa15-d457bad1a171 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.771376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcbed2-32f9-4dec-aa15-d457bad1a171 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.772015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcbed2-32f9-4dec-aa15-d457bad1a171 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:54.773458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcbed2-32f9-4dec-aa15-d457bad1a171 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:54.824687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:54.828022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0d99f-93a4-42b1-b322-fd89612f49ea None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:54.861415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d2a36-d26a-48a1-86d5-f27a1de4451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:54.861956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d2a36-d26a-48a1-86d5-f27a1de4451e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:54.862235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d2a36-d26a-48a1-86d5-f27a1de4451e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:54.862506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d2a36-d26a-48a1-86d5-f27a1de4451e None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:54.863164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d2a36-d26a-48a1-86d5-f27a1de4451e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:54.863467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d2a36-d26a-48a1-86d5-f27a1de4451e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:54.865090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d2a36-d26a-48a1-86d5-f27a1de4451e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:54.874742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 504/1015] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:04:54 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:54.874986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 512/1016] 10.222.0.30 () {56 vars in 1112 bytes} [Sun Aug 7 12:04:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:54.875899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcbed2-32f9-4dec-aa15-d457bad1a171 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:54.904020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:54.985715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcbed2-32f9-4dec-aa15-d457bad1a171 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:55.024119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bb0dc-33cf-4158-81e6-a44f02e0528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:55.024905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bb0dc-33cf-4158-81e6-a44f02e0528d None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:55.025185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bb0dc-33cf-4158-81e6-a44f02e0528d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:55.025454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bb0dc-33cf-4158-81e6-a44f02e0528d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:55.026103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bb0dc-33cf-4158-81e6-a44f02e0528d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:55.027501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bb0dc-33cf-4158-81e6-a44f02e0528d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:55.102968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 505/1017] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:04:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:04:55.688265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1baf-9dc0-4ec3-9bbd-b70cf0509b21 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:55.688831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1baf-9dc0-4ec3-9bbd-b70cf0509b21 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:55.688831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1baf-9dc0-4ec3-9bbd-b70cf0509b21 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:55.690329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 513/1018] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:55.695281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1a3c1-be30-4c29-a5a5-fb618ffebacc None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:55.696199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1a3c1-be30-4c29-a5a5-fb618ffebacc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:55.696390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1a3c1-be30-4c29-a5a5-fb618ffebacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:55.714719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d4c1a3c1-be30-4c29-a5a5-fb618ffebacc None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:55.715939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d4c1a3c1-be30-4c29-a5a5-fb618ffebacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:55.789183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:55.795963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 506/1019] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:55 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:55.818347 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1baf-9dc0-4ec3-9bbd-b70cf0509b21 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:55.923755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1baf-9dc0-4ec3-9bbd-b70cf0509b21 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:55.956735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4811c7-0d13-4af2-bed4-eb56bae35d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:55.957301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4811c7-0d13-4af2-bed4-eb56bae35d6e None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:55.957583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4811c7-0d13-4af2-bed4-eb56bae35d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:55.957858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4811c7-0d13-4af2-bed4-eb56bae35d6e None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:55.958499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4811c7-0d13-4af2-bed4-eb56bae35d6e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:55.959891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4811c7-0d13-4af2-bed4-eb56bae35d6e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:56.059090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 514/1020] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:04:55 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:04:56.939624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d552fc-d172-49d2-8dc7-2c35c4c81fe3 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:56.940181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d552fc-d172-49d2-8dc7-2c35c4c81fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:56.940181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d552fc-d172-49d2-8dc7-2c35c4c81fe3 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:56.941658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 507/1021] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:56.947271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420ea5-9f9d-41af-93f8-8caefd534d09 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:56.947709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420ea5-9f9d-41af-93f8-8caefd534d09 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:56.947884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420ea5-9f9d-41af-93f8-8caefd534d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:56.965805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8c420ea5-9f9d-41af-93f8-8caefd534d09 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:56.967017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8c420ea5-9f9d-41af-93f8-8caefd534d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:56.985515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:57.042571 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 515/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:57.066807 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d552fc-d172-49d2-8dc7-2c35c4c81fe3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:57.180047 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d552fc-d172-49d2-8dc7-2c35c4c81fe3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:57.192695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c339d-bc52-4dc4-afd2-16030d712087 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:57.192929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c339d-bc52-4dc4-afd2-16030d712087 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:57.193183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c339d-bc52-4dc4-afd2-16030d712087 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:57.194526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 516/1023] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:57.200054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf646f8a-29ac-4ce1-8b7f-4d706b8163cd None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:57.200342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf646f8a-29ac-4ce1-8b7f-4d706b8163cd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:57.200617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf646f8a-29ac-4ce1-8b7f-4d706b8163cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:57.216119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b135ed3-5799-4393-a211-b1b29b6d8ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:57.216747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135ed3-5799-4393-a211-b1b29b6d8ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:57.217025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135ed3-5799-4393-a211-b1b29b6d8ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:57.217297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135ed3-5799-4393-a211-b1b29b6d8ed4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:57.218369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bf646f8a-29ac-4ce1-8b7f-4d706b8163cd None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:57.218477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b135ed3-5799-4393-a211-b1b29b6d8ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:57.219651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bf646f8a-29ac-4ce1-8b7f-4d706b8163cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:57.219828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b135ed3-5799-4393-a211-b1b29b6d8ed4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:57.221542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 508/1024] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:57.227463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b135ed3-5799-4393-a211-b1b29b6d8ed4 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:57.295040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 517/1025] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:57.298722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:57.313654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf646f8a-29ac-4ce1-8b7f-4d706b8163cd None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:57.333063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b135ed3-5799-4393-a211-b1b29b6d8ed4 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:57.372084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a14ce1-c099-4454-b119-340ffea8fa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:57.372689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a14ce1-c099-4454-b119-340ffea8fa5d None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:57.372962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a14ce1-c099-4454-b119-340ffea8fa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:57.373235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a14ce1-c099-4454-b119-340ffea8fa5d None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:57.373872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a14ce1-c099-4454-b119-340ffea8fa5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:57.375315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a14ce1-c099-4454-b119-340ffea8fa5d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:57.398605 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:57.418706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf646f8a-29ac-4ce1-8b7f-4d706b8163cd None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:57.453548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a707648-51c0-4a57-b683-cdd20edc0fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:57.454108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a707648-51c0-4a57-b683-cdd20edc0fbd None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:57.454387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a707648-51c0-4a57-b683-cdd20edc0fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:57.454662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a707648-51c0-4a57-b683-cdd20edc0fbd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:57.460206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 509/1026] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:04:57 2022] POST /identity/v3/users => generated 285 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:04:57.465598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a707648-51c0-4a57-b683-cdd20edc0fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-9a707648-51c0-4a57-b683-cdd20edc0fbd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:57.470720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:57.472177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:04:57.473277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:57.481718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 518/1027] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 12:04:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:57.482797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a14ce1-c099-4454-b119-340ffea8fa5d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:57.594192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a14ce1-c099-4454-b119-340ffea8fa5d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:57.619440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:57.630597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:57.631239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:57.637092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:57.637092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:57.637092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:57.637092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:57.637092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:57.642856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 510/1028] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 12:04:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:57.648601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a707648-51c0-4a57-b683-cdd20edc0fbd None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:57.760787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a707648-51c0-4a57-b683-cdd20edc0fbd None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:57.796931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:57.797845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:57.798332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:57.798778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:57.807741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:57.810442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:57.811926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:57.826496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:57.827746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:57.829391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:57.829391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:57.829391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:57.829391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 519/1029] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:57.833547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:57.938193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71333a4-0069-48b0-8060-d932a6e94833 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:57.972904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e21e6-1751-4b2d-bd83-ee788cf1478f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:57.973708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e21e6-1751-4b2d-bd83-ee788cf1478f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:57.973996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e21e6-1751-4b2d-bd83-ee788cf1478f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:57.974271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e21e6-1751-4b2d-bd83-ee788cf1478f None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:57.974914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e21e6-1751-4b2d-bd83-ee788cf1478f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:57.975258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e21e6-1751-4b2d-bd83-ee788cf1478f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:57.976619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e21e6-1751-4b2d-bd83-ee788cf1478f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:57.994398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 511/1030] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:57.995632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.071644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ed1fa-b1c4-48a8-bd0c-35628e0c5269 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.071943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ed1fa-b1c4-48a8-bd0c-35628e0c5269 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.072109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ed1fa-b1c4-48a8-bd0c-35628e0c5269 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.073590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 512/1031] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:04:58.078404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674ad4-7b59-4b99-9c71-bc5f0097c009 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.078690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674ad4-7b59-4b99-9c71-bc5f0097c009 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.078962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674ad4-7b59-4b99-9c71-bc5f0097c009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.097867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0e674ad4-7b59-4b99-9c71-bc5f0097c009 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:58.099078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0e674ad4-7b59-4b99-9c71-bc5f0097c009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:58.099432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74197b-aa73-485e-a999-8e4d3fd168fc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.141530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:58.142158 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.142440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.142717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.144022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:58.151826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:58.153358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:04:58.164595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:58.165620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:58.166593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 520/1032] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:04:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:58.171712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.174652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 513/1033] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:04:58.197197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e674ad4-7b59-4b99-9c71-bc5f0097c009 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.228577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:58.280816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3a354-8cb7-4a43-abd4-3be1904ed52b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.308507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e674ad4-7b59-4b99-9c71-bc5f0097c009 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.316734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:58.317344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.317625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.317900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.318638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:58.318946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:58.320412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:58.343307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 521/1034] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:58.344292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.360574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:58.361581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.361877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.362156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] PATH_INFO: `/v3/services/image` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.362744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:58.364129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:58.372924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:58.373928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:58.374801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 514/1035] 10.222.0.30 () {56 vars in 1105 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:58.379933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.421843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:58.456220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806b632-b6d5-43b6-bebe-4a01f5d93029 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.467697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:58.487255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e9c6-5e8d-401c-80a8-cc2ddec276ac None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.494218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50c7b2-d57f-4a2d-862f-c0af5d179c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:58.494880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50c7b2-d57f-4a2d-862f-c0af5d179c55 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.495629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50c7b2-d57f-4a2d-862f-c0af5d179c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.496202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50c7b2-d57f-4a2d-862f-c0af5d179c55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.526794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:58.527421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.527707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.527984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.528606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] RBAC: Adding query filter params (name=image) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:58.528907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:58.530265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:58.536080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 515/1036] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:58.542210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.573721 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50c7b2-d57f-4a2d-862f-c0af5d179c55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:58.575210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50c7b2-d57f-4a2d-862f-c0af5d179c55 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:58.652146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec540e-6b4e-4155-b266-4f06c2491f5f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.653879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:58.687823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 522/1037] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 12:04:58.688977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50c7b2-d57f-4a2d-862f-c0af5d179c55 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.695178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:58.696346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.696624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.696893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.697993 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] RBAC: Adding query filter params (type=image) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:58.698302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:58.700061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:58.700924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:58.710094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 516/1038] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:58.712063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.799705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50c7b2-d57f-4a2d-862f-c0af5d179c55 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.817325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff55fb-968e-4003-abb6-183adcfb9a37 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.838290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:58.838874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.839184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.839459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.849661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:58.851131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:58.853300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:58.853893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:58.854388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:58.854507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:58.855137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] RBAC: Adding query filter params (interface=public, service_id=bec716949aa6453f8fb4bdafd295779d, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:58.855443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:58.856785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:58.862065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 517/1039] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/endpoints?service_id=bec716949aa6453f8fb4bdafd295779d&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:58.867642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:04:58.868005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:58.868951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:04:58.869895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:04:58.869895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:04:58.869895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:58.869895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 523/1040] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:58.874479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:58.902053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:58.950747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:58.984388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da2427-a5c0-4eb1-870c-295992f2c3fd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:58.993196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791838-2804-409d-967b-ee0b2baec20f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:59.021848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a974c-26a5-4eae-8a3f-1187895d866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:59.022456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a974c-26a5-4eae-8a3f-1187895d866d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:59.022733 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a974c-26a5-4eae-8a3f-1187895d866d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:59.023187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a974c-26a5-4eae-8a3f-1187895d866d None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:59.023824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a974c-26a5-4eae-8a3f-1187895d866d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:59.025176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a974c-26a5-4eae-8a3f-1187895d866d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:59.030767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:59.031912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 518/1041] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:04:59.032135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:59.032419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:59.032691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:59.033401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:59.033707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:59.035124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:59.056503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 524/1042] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:04:59.062076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a974c-26a5-4eae-8a3f-1187895d866d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:59.169887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:59.176742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a974c-26a5-4eae-8a3f-1187895d866d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:59.214704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:59.215293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:59.215574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:59.215845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:59.223679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:59.225122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:04:59.237177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:59.238312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:04:59.239396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 519/1043] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:04:59.245098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:59.346389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d7d7-35e4-4600-8d51-1d267753bec4 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:59.370273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:59.381912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791935b6-6a70-4294-8eb4-001c2ea8e158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:59.382541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791935b6-6a70-4294-8eb4-001c2ea8e158 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:59.382821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791935b6-6a70-4294-8eb4-001c2ea8e158 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:59.383649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791935b6-6a70-4294-8eb4-001c2ea8e158 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:59.384338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791935b6-6a70-4294-8eb4-001c2ea8e158 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:59.384640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791935b6-6a70-4294-8eb4-001c2ea8e158 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:59.386110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791935b6-6a70-4294-8eb4-001c2ea8e158 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:59.398237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 525/1044] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:04:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:59.404572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:59.506184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf595ed-690e-4bfb-bec5-f29335a6edfa None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:59.541365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cd9bf-ec6c-42e7-9f41-3106666170c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:59.541968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd9bf-ec6c-42e7-9f41-3106666170c7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:59.542245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd9bf-ec6c-42e7-9f41-3106666170c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:59.542522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd9bf-ec6c-42e7-9f41-3106666170c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:59.543270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd9bf-ec6c-42e7-9f41-3106666170c7 None admin] RBAC: Adding query filter params (role.id=93f3167474ff4071a8c82c01f6fd3f1c, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=cb2075e9fff3459881c13f81456ab198) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:04:59.543577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd9bf-ec6c-42e7-9f41-3106666170c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:04:59.544986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd9bf-ec6c-42e7-9f41-3106666170c7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:04:59.550825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 520/1045] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:04:59 2022] GET /identity/v3/role_assignments?user.id=cb2075e9fff3459881c13f81456ab198&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=93f3167474ff4071a8c82c01f6fd3f1c => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:04:59.595123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0648b4a-f098-4425-9e3c-acf729e928f5 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:59.595508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0648b4a-f098-4425-9e3c-acf729e928f5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:59.595675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0648b4a-f098-4425-9e3c-acf729e928f5 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:59.597263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 526/1046] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:04:59.602968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e3bb-cd2e-4e5b-81c1-f3548a98efe7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:59.603317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e3bb-cd2e-4e5b-81c1-f3548a98efe7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:59.603568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e3bb-cd2e-4e5b-81c1-f3548a98efe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:59.623653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d081e3bb-cd2e-4e5b-81c1-f3548a98efe7 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:04:59.624882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d081e3bb-cd2e-4e5b-81c1-f3548a98efe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:04:59.640386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:04:59.699595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 521/1047] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:04:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:04:59.722680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0648b4a-f098-4425-9e3c-acf729e928f5 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:59.829782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0648b4a-f098-4425-9e3c-acf729e928f5 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:04:59.863486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf812f3f-5725-495a-b4d1-5807e1d2b140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:04:59.864108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf812f3f-5725-495a-b4d1-5807e1d2b140 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:04:59.864391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf812f3f-5725-495a-b4d1-5807e1d2b140 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:04:59.864663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf812f3f-5725-495a-b4d1-5807e1d2b140 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:04:59.875791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf812f3f-5725-495a-b4d1-5807e1d2b140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-cf812f3f-5725-495a-b4d1-5807e1d2b140 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:04:59.880398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:04:59.881533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:04:59.882618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:04:59.884478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 527/1048] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:04:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:04:59.887976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081e3bb-cd2e-4e5b-81c1-f3548a98efe7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:04:59.988544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081e3bb-cd2e-4e5b-81c1-f3548a98efe7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:00.022715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:00.023344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:00.023632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:00.023907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:00.024673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:00.024979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:00.026323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:00.038921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 522/1049] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:00.041081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf812f3f-5725-495a-b4d1-5807e1d2b140 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:00.085719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:00.145499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf812f3f-5725-495a-b4d1-5807e1d2b140 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:00.183245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0be6d-db6b-4467-aa9c-8b443b6d55bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:00.183882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0be6d-db6b-4467-aa9c-8b443b6d55bb None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:00.184163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0be6d-db6b-4467-aa9c-8b443b6d55bb None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:00.184438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0be6d-db6b-4467-aa9c-8b443b6d55bb None admin] PATH_INFO: `/v3/users/7e47b469355e48d3b37729c8e3d84bb0` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:00.198492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0be6d-db6b-4467-aa9c-8b443b6d55bb None admin] RBAC: Authorizing `identity:get_user(user_id=7e47b469355e48d3b37729c8e3d84bb0)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:00.199973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0be6d-db6b-4467-aa9c-8b443b6d55bb None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:00.201793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 528/1050] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:00 2022] GET /identity/v3/users/7e47b469355e48d3b37729c8e3d84bb0 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:00.207346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:00.220269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:00.309306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea20a2-225e-475a-a929-058c8981567e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:00.342925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e66abd-8ecf-4b5b-85a4-fff13dc6f1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:00.343480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66abd-8ecf-4b5b-85a4-fff13dc6f1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:00.343758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66abd-8ecf-4b5b-85a4-fff13dc6f1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:00.344029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66abd-8ecf-4b5b-85a4-fff13dc6f1c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:00.410637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e66abd-8ecf-4b5b-85a4-fff13dc6f1c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:00.412119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e66abd-8ecf-4b5b-85a4-fff13dc6f1c9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:00.422919 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:00.511671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 523/1051] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:05:00.517652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0be6d-db6b-4467-aa9c-8b443b6d55bb None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:00.614466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:00.618439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0be6d-db6b-4467-aa9c-8b443b6d55bb None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:00.653761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd70b4-44e3-4aa6-a1ca-0e0863104cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:00.654418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd70b4-44e3-4aa6-a1ca-0e0863104cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:00.654696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd70b4-44e3-4aa6-a1ca-0e0863104cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:00.654963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd70b4-44e3-4aa6-a1ca-0e0863104cc8 None admin] PATH_INFO: `/v3/projects/69a97ec1fee7433aae3ef0397c97c909` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:00.656094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd70b4-44e3-4aa6-a1ca-0e0863104cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=69a97ec1fee7433aae3ef0397c97c909)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:00.657585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd70b4-44e3-4aa6-a1ca-0e0863104cc8 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:00.664123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 529/1052] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:00 2022] GET /identity/v3/projects/69a97ec1fee7433aae3ef0397c97c909 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:00.665749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e66abd-8ecf-4b5b-85a4-fff13dc6f1c9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:00.765350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e66abd-8ecf-4b5b-85a4-fff13dc6f1c9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:00.776328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:00.799802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb3499-d4a4-477d-bea7-141f11b889a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:00.800443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb3499-d4a4-477d-bea7-141f11b889a9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:00.800914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb3499-d4a4-477d-bea7-141f11b889a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:00.801224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb3499-d4a4-477d-bea7-141f11b889a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:00.802095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb3499-d4a4-477d-bea7-141f11b889a9 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=69a97ec1fee7433aae3ef0397c97c909, user.id=7e47b469355e48d3b37729c8e3d84bb0) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:00.802654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb3499-d4a4-477d-bea7-141f11b889a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:00.804096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb3499-d4a4-477d-bea7-141f11b889a9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:00.815030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 524/1053] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:00 2022] GET /identity/v3/role_assignments?user.id=7e47b469355e48d3b37729c8e3d84bb0&scope.project.id=69a97ec1fee7433aae3ef0397c97c909&role.id=f72d3388f13149b483355331380f040b => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:01.088455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3ef6d-a3c5-41ba-ae6f-1cf35d35c167 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.088974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3ef6d-a3c5-41ba-ae6f-1cf35d35c167 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.089054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3ef6d-a3c5-41ba-ae6f-1cf35d35c167 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.090524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 530/1054] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:01.095774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eda0f-0f77-4f83-b19b-b028e516457a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.096202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eda0f-0f77-4f83-b19b-b028e516457a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.096389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eda0f-0f77-4f83-b19b-b028e516457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.118632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-367eda0f-0f77-4f83-b19b-b028e516457a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:01.119866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-367eda0f-0f77-4f83-b19b-b028e516457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:01.191052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 525/1055] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:01.212504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3ef6d-a3c5-41ba-ae6f-1cf35d35c167 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:01.318708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3ef6d-a3c5-41ba-ae6f-1cf35d35c167 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:01.354783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:01.355370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.355645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.355935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] PATH_INFO: `/v3/services/image` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.356490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:01.357840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:01.366706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:01.367954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:01.374085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 531/1056] 10.222.0.30 () {56 vars in 1105 bytes} [Sun Aug 7 12:05:01 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:01.376284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367eda0f-0f77-4f83-b19b-b028e516457a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:01.479505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367eda0f-0f77-4f83-b19b-b028e516457a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:01.481898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:01.514724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde2811-63c1-46fb-abae-da1ab3e10faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:01.515419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2811-63c1-46fb-abae-da1ab3e10faa None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.515691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2811-63c1-46fb-abae-da1ab3e10faa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.515967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2811-63c1-46fb-abae-da1ab3e10faa None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.516603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde2811-63c1-46fb-abae-da1ab3e10faa None admin] RBAC: Adding query filter params (name=image) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:01.516902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde2811-63c1-46fb-abae-da1ab3e10faa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:01.518617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde2811-63c1-46fb-abae-da1ab3e10faa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:01.529282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 526/1057] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:01.530917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:01.562986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:01.589836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765f8fb-cd2f-49fd-9cf2-d63bc66ab796 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.590151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765f8fb-cd2f-49fd-9cf2-d63bc66ab796 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.590514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765f8fb-cd2f-49fd-9cf2-d63bc66ab796 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.592135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 527/1058] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:05:01.597410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f723e-dce3-478b-a8e3-bf9290d4c6c3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.597688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f723e-dce3-478b-a8e3-bf9290d4c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.597958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f723e-dce3-478b-a8e3-bf9290d4c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.617279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9e9f723e-dce3-478b-a8e3-bf9290d4c6c3 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:01.619332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9e9f723e-dce3-478b-a8e3-bf9290d4c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:01.644754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f6038-0aa5-4e30-89d1-431901551f24 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:01.681233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:01.681830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.682110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.682389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.683037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] RBAC: Adding query filter params (type=image) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:01.683343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:01.684710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:01.687825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 528/1059] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:01.692873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 532/1060] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:01.698737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f723e-dce3-478b-a8e3-bf9290d4c6c3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:01.712450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:01.802427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:01.806604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f723e-dce3-478b-a8e3-bf9290d4c6c3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:01.817627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:01.825113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da128b72-80f0-4bee-86a0-0a56702fa351 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:01.844390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bfdbb-59c0-4d44-b610-1fae752bb569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:01.844994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bfdbb-59c0-4d44-b610-1fae752bb569 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.845274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bfdbb-59c0-4d44-b610-1fae752bb569 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.845543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bfdbb-59c0-4d44-b610-1fae752bb569 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.846207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bfdbb-59c0-4d44-b610-1fae752bb569 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:01.847616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bfdbb-59c0-4d44-b610-1fae752bb569 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:01.863609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b84b5-762a-4c55-bd12-1a7b47520b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:01.864231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b84b5-762a-4c55-bd12-1a7b47520b6f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:01.864507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b84b5-762a-4c55-bd12-1a7b47520b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:01.864775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b84b5-762a-4c55-bd12-1a7b47520b6f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:01.876333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b84b5-762a-4c55-bd12-1a7b47520b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-0b7b84b5-762a-4c55-bd12-1a7b47520b6f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:01.881007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:01.882152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:01.884689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 533/1061] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 12:05:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:01.888918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b84b5-762a-4c55-bd12-1a7b47520b6f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:01.935434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 529/1062] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:05:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:05:02.002215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b84b5-762a-4c55-bd12-1a7b47520b6f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:02.046926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:02.047573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:02.047854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:02.048129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:02.048798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:02.049102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:02.050488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:02.061278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 534/1063] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 12:05:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:02.067680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bfdbb-59c0-4d44-b610-1fae752bb569 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:02.185021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bfdbb-59c0-4d44-b610-1fae752bb569 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:02.220018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:02.220641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:02.220917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:02.221184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:02.231530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:02.232958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:05:02.247791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:02.248797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:05:02.249734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:05:02.249734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:02.249734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:02.249734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 530/1064] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:02.255715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:02.272236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:02.358372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554cfff-cf0d-4971-ba5b-a10e36952d34 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:02.392548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:02.393148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:02.393424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:02.393695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:02.394334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:02.394635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:02.396024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:02.412532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 535/1065] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:02.413850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:02.423699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:02.517634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5de1b1-545d-4023-8b4a-59842a5f7afd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:02.551857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:02.552483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:02.552763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:02.553036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:02.561386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:02.562837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:05:02.574116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:02.575415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:05:02.576393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:02.579316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 531/1066] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:05:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:02.580862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:02.658884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:02.684095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccf721-8cc9-4714-b47f-b8d215213b16 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:02.718290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8901dc-c121-4af0-a5af-2cf30ae05098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:02.719081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8901dc-c121-4af0-a5af-2cf30ae05098 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:02.719460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8901dc-c121-4af0-a5af-2cf30ae05098 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:02.719832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8901dc-c121-4af0-a5af-2cf30ae05098 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:02.720653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8901dc-c121-4af0-a5af-2cf30ae05098 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:02.721059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8901dc-c121-4af0-a5af-2cf30ae05098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:02.722566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8901dc-c121-4af0-a5af-2cf30ae05098 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:02.745398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 536/1067] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:05:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:02.746811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:02.812180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:02.820375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f5ea9-1ad6-403f-b105-cd621a95bbf3 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:02.821276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f5ea9-1ad6-403f-b105-cd621a95bbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:02.821567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f5ea9-1ad6-403f-b105-cd621a95bbf3 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:02.822992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 537/1068] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:02.827756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c42fa-70a8-49df-a649-1ef9dd4b9b7d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:02.828047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c42fa-70a8-49df-a649-1ef9dd4b9b7d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:02.828324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c42fa-70a8-49df-a649-1ef9dd4b9b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:02.845874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e40c42fa-70a8-49df-a649-1ef9dd4b9b7d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:02.847090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e40c42fa-70a8-49df-a649-1ef9dd4b9b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:02.848747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5381b-f487-4a75-9d46-91b04726f01a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:02.883664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:02.884435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:02.884778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:02.885055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:02.895276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:02.896693 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:05:02.910446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:02.911631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:05:02.912546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:05:02.912546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:02.912546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:02.912546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 532/1069] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:05:02.917743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:02.921495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 538/1070] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:05:02.956200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c42fa-70a8-49df-a649-1ef9dd4b9b7d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:03.038899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:03.050082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca7a90-ec7f-476d-a53f-c66b18168626 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.068118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:03.079225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c42fa-70a8-49df-a649-1ef9dd4b9b7d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.088753 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.089773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.090067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.090347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.091032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:03.091739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:03.093455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:03.118379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 533/1071] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:03.119980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:03.127094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4072ea9-eb8a-4f3d-81f6-b3a9c8fa29b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.127921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072ea9-eb8a-4f3d-81f6-b3a9c8fa29b1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.128383 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072ea9-eb8a-4f3d-81f6-b3a9c8fa29b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.128822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072ea9-eb8a-4f3d-81f6-b3a9c8fa29b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.140658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4072ea9-eb8a-4f3d-81f6-b3a9c8fa29b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-c4072ea9-eb8a-4f3d-81f6-b3a9c8fa29b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:03.147163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:03.148355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:03.149738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:03.152784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 539/1072] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:05:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:05:03.158980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4072ea9-eb8a-4f3d-81f6-b3a9c8fa29b1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:03.228516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f105e63-ed62-4a29-a503-fdec571e7c90 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.267306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.267931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.268207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.268476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.268992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4072ea9-eb8a-4f3d-81f6-b3a9c8fa29b1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.276722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:03.278177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:05:03.289354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:03.291401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:03.292709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 534/1073] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:05:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:03.299445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:03.305736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:03.322626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.323500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.323762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.324210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.325311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:03.325798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:03.328181 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:03.342666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 540/1074] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:03.348627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:03.362117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:03.423735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f66b3a-0f9e-484a-a779-2b2cb53a75fb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.457095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1302c0e-abe5-487a-8884-8748b20b2633 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.458247 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.458963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.459266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.459551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.460355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:03.460659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:03.463110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:03.476120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 535/1075] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:05:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:03.481992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:03.497151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f0d8-70ad-4f49-aa40-427331b3e403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.497896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f0d8-70ad-4f49-aa40-427331b3e403 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.498226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f0d8-70ad-4f49-aa40-427331b3e403 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.499939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f0d8-70ad-4f49-aa40-427331b3e403 None admin] PATH_INFO: `/v3/users/7e47b469355e48d3b37729c8e3d84bb0` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.499939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f0d8-70ad-4f49-aa40-427331b3e403 None admin] RBAC: Authorizing `identity:get_user(user_id=7e47b469355e48d3b37729c8e3d84bb0)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:03.501619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f0d8-70ad-4f49-aa40-427331b3e403 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:03.507913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 541/1076] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:03 2022] GET /identity/v3/users/7e47b469355e48d3b37729c8e3d84bb0 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:03.509771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f0d8-70ad-4f49-aa40-427331b3e403 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:03.568614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:03.590627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:03.595289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ff080-bf97-4626-8f78-528bfabb6ff7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.618991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f0d8-70ad-4f49-aa40-427331b3e403 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.631931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a41e7-0b82-48b2-8500-8c40bc48f08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.632587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a41e7-0b82-48b2-8500-8c40bc48f08f None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.632866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a41e7-0b82-48b2-8500-8c40bc48f08f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.633671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a41e7-0b82-48b2-8500-8c40bc48f08f None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/cb2075e9fff3459881c13f81456ab198/roles/93f3167474ff4071a8c82c01f6fd3f1c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.636062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a41e7-0b82-48b2-8500-8c40bc48f08f None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=cb2075e9fff3459881c13f81456ab198, role_id=93f3167474ff4071a8c82c01f6fd3f1c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:03.637482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a41e7-0b82-48b2-8500-8c40bc48f08f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:03.656319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7aea-2df7-40b2-955b-985be57312ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.656947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7aea-2df7-40b2-955b-985be57312ce None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.657232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7aea-2df7-40b2-955b-985be57312ce None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.657508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7aea-2df7-40b2-955b-985be57312ce None admin] PATH_INFO: `/v3/projects/69a97ec1fee7433aae3ef0397c97c909` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.658631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7aea-2df7-40b2-955b-985be57312ce None admin] RBAC: Authorizing `identity:get_project(project_id=69a97ec1fee7433aae3ef0397c97c909)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:03.660130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7aea-2df7-40b2-955b-985be57312ce None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:03.662021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 542/1077] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:03 2022] GET /identity/v3/projects/69a97ec1fee7433aae3ef0397c97c909 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:03.667716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7aea-2df7-40b2-955b-985be57312ce None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:03.717572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 536/1078] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:05:03 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/cb2075e9fff3459881c13f81456ab198/roles/93f3167474ff4071a8c82c01f6fd3f1c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:05:03.775062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7aea-2df7-40b2-955b-985be57312ce None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:03.797328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:03.812516 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393634c-8e0d-4d3d-b713-cd90ef50f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:03.813186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393634c-8e0d-4d3d-b713-cd90ef50f422 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:03.813488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393634c-8e0d-4d3d-b713-cd90ef50f422 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:03.813766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393634c-8e0d-4d3d-b713-cd90ef50f422 None admin] PATH_INFO: `/v3/projects/69a97ec1fee7433aae3ef0397c97c909/users/7e47b469355e48d3b37729c8e3d84bb0/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:03.816012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393634c-8e0d-4d3d-b713-cd90ef50f422 None admin] RBAC: Authorizing `identity:create_grant(project_id=69a97ec1fee7433aae3ef0397c97c909, user_id=7e47b469355e48d3b37729c8e3d84bb0, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:03.817476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393634c-8e0d-4d3d-b713-cd90ef50f422 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:03.917388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 543/1079] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:05:03 2022] PUT /identity/v3/projects/69a97ec1fee7433aae3ef0397c97c909/users/7e47b469355e48d3b37729c8e3d84bb0/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:05:04.016894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f06b7b-5228-4ce0-a7e5-24c38a6e30bb None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:04.017232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f06b7b-5228-4ce0-a7e5-24c38a6e30bb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:04.017452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f06b7b-5228-4ce0-a7e5-24c38a6e30bb None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:04.018905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 537/1080] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:04.025062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e794e7-4389-4b6c-bdb3-9b6fc9d3a002 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:04.025384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e794e7-4389-4b6c-bdb3-9b6fc9d3a002 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:04.025615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e794e7-4389-4b6c-bdb3-9b6fc9d3a002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:04.044683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c0e794e7-4389-4b6c-bdb3-9b6fc9d3a002 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:04.045947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c0e794e7-4389-4b6c-bdb3-9b6fc9d3a002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:04.127475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 544/1081] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:04.150138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f06b7b-5228-4ce0-a7e5-24c38a6e30bb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:04.256770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f06b7b-5228-4ce0-a7e5-24c38a6e30bb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:04.291668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02dfc6-b3cb-4ce1-9e35-c73e9210302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:04.292268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02dfc6-b3cb-4ce1-9e35-c73e9210302e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:04.292543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02dfc6-b3cb-4ce1-9e35-c73e9210302e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:04.292813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02dfc6-b3cb-4ce1-9e35-c73e9210302e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:04.318916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:04.361638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02dfc6-b3cb-4ce1-9e35-c73e9210302e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:04.363095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02dfc6-b3cb-4ce1-9e35-c73e9210302e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:04.468721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 538/1082] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 12:05:04.469907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e794e7-4389-4b6c-bdb3-9b6fc9d3a002 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:04.568448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e794e7-4389-4b6c-bdb3-9b6fc9d3a002 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:04.580584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:04.603892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12a38e-d22f-447b-afee-9ab175601f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:04.604517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12a38e-d22f-447b-afee-9ab175601f33 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:04.604791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12a38e-d22f-447b-afee-9ab175601f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:04.605057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12a38e-d22f-447b-afee-9ab175601f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:04.615659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12a38e-d22f-447b-afee-9ab175601f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:04.617109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12a38e-d22f-447b-afee-9ab175601f33 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6a12a38e-d22f-447b-afee-9ab175601f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:05:04.631482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:04.632565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:05:04.634156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:05:04.634156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:04.634156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:04.634156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 545/1083] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:04.640524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02dfc6-b3cb-4ce1-9e35-c73e9210302e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:04.698809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:04.741153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02dfc6-b3cb-4ce1-9e35-c73e9210302e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:04.774689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12186375-5ce3-4b25-99a0-a831b0e73988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:04.775315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12186375-5ce3-4b25-99a0-a831b0e73988 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:04.775592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12186375-5ce3-4b25-99a0-a831b0e73988 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:04.775860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12186375-5ce3-4b25-99a0-a831b0e73988 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:04.776495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12186375-5ce3-4b25-99a0-a831b0e73988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:04.776792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12186375-5ce3-4b25-99a0-a831b0e73988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:04.778158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12186375-5ce3-4b25-99a0-a831b0e73988 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:04.799044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 539/1084] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:05.723398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aed0f-b44f-4375-bd1b-fea5ff17a1ec None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:05.724217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aed0f-b44f-4375-bd1b-fea5ff17a1ec None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:05.724377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aed0f-b44f-4375-bd1b-fea5ff17a1ec None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:05.725778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 546/1085] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:05.731026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32c82-3def-4720-8d39-52b23b228aec None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:05.731437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32c82-3def-4720-8d39-52b23b228aec None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:05.731580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32c82-3def-4720-8d39-52b23b228aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:05.749924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9de32c82-3def-4720-8d39-52b23b228aec None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:05.751109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9de32c82-3def-4720-8d39-52b23b228aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:05.815558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 540/1086] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:05.838957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aed0f-b44f-4375-bd1b-fea5ff17a1ec None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:05.935687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e8985-71ed-4495-bd5d-b253cd107597 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:05.936070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e8985-71ed-4495-bd5d-b253cd107597 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:05.936346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e8985-71ed-4495-bd5d-b253cd107597 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:05.937683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 541/1087] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:05:05.939735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aed0f-b44f-4375-bd1b-fea5ff17a1ec None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:05.942423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2056-ce87-4154-bbce-2e636863e0a9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:05.942701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2056-ce87-4154-bbce-2e636863e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:05.942970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2056-ce87-4154-bbce-2e636863e0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:05.960773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-060b2056-ce87-4154-bbce-2e636863e0a9 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:05.962170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-060b2056-ce87-4154-bbce-2e636863e0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:05.973869 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09024d-fbfa-4cc3-b191-822badf5b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:05.974425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09024d-fbfa-4cc3-b191-822badf5b816 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:05.974707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09024d-fbfa-4cc3-b191-822badf5b816 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:05.974978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09024d-fbfa-4cc3-b191-822badf5b816 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:05.986362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09024d-fbfa-4cc3-b191-822badf5b816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-2f09024d-fbfa-4cc3-b191-822badf5b816 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:05.991343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:05.992890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:05.993772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 547/1088] 10.222.0.30 () {56 vars in 1120 bytes} [Sun Aug 7 12:05:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:05.999066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09024d-fbfa-4cc3-b191-822badf5b816 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.005885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.022751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.029116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 542/1089] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:05:06.052389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b2056-ce87-4154-bbce-2e636863e0a9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.097334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09024d-fbfa-4cc3-b191-822badf5b816 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.134546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.135178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.135453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.135723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.136381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:06.136678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.138041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.145790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 548/1090] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 12:05:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:06.151477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.152804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b2056-ce87-4154-bbce-2e636863e0a9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.186417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35cf91-ebea-40ad-ab47-ad12e4b54630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.187070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35cf91-ebea-40ad-ab47-ad12e4b54630 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.187354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35cf91-ebea-40ad-ab47-ad12e4b54630 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.187626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35cf91-ebea-40ad-ab47-ad12e4b54630 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.198386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35cf91-ebea-40ad-ab47-ad12e4b54630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1a35cf91-ebea-40ad-ab47-ad12e4b54630 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:06.203118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:06.204086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:06.204952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 543/1091] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:05:06.210178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35cf91-ebea-40ad-ab47-ad12e4b54630 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.235845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.251742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.253431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1e53-03b2-470c-8381-7325fce32888 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.287140 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.287761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.288043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.288318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.299450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.300902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.312470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35cf91-ebea-40ad-ab47-ad12e4b54630 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:05:06.314676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:06.315748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:05:06.317141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:05:06.317141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:06.317141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:06.317141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 549/1092] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:05:06.321393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.346286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.346885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.347188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.347458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.348110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:06.348407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.349755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.358189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 544/1093] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:06.363889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.419998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5729a-b209-424e-a38e-a9325b697703 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.452975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.453569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.453846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.454114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.454740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:06.455068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.456437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.458617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.459120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.462412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae2019-9e7e-476e-8311-3eafe839d472 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.470075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 550/1094] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:06.475785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.499849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0340b3-d323-4325-a973-1b546bf47c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.499849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0340b3-d323-4325-a973-1b546bf47c5b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.499849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0340b3-d323-4325-a973-1b546bf47c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.499849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0340b3-d323-4325-a973-1b546bf47c5b None admin] PATH_INFO: `/v3/users/7e47b469355e48d3b37729c8e3d84bb0` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.500290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0340b3-d323-4325-a973-1b546bf47c5b None admin] RBAC: Authorizing `identity:get_user(user_id=7e47b469355e48d3b37729c8e3d84bb0)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.500709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0340b3-d323-4325-a973-1b546bf47c5b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.502279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 545/1095] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/users/7e47b469355e48d3b37729c8e3d84bb0 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:06.508130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0340b3-d323-4325-a973-1b546bf47c5b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.574843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f40d21-7901-4b6b-8163-1fb567c50acd None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.606112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0340b3-d323-4325-a973-1b546bf47c5b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.609583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.610216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.610498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.610773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.618665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.620131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:05:06.631357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:06.632343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:06.633322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 551/1096] 10.222.0.30 () {56 vars in 1093 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:05:06.638249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.642675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e686644-8ce9-4f82-9b99-4567ffedf285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.643305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e686644-8ce9-4f82-9b99-4567ffedf285 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.643586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e686644-8ce9-4f82-9b99-4567ffedf285 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.643858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e686644-8ce9-4f82-9b99-4567ffedf285 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.678526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.693801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.710718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e686644-8ce9-4f82-9b99-4567ffedf285 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.712484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e686644-8ce9-4f82-9b99-4567ffedf285 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.736893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067dea-858d-4cbb-9000-c20bdbd48409 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.770800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.771343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.771621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.771892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.772603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:06.772905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.774493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.791889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 552/1097] 10.222.0.30 () {56 vars in 1133 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:06.797452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.814655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 546/1098] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 12:05:06.820486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e686644-8ce9-4f82-9b99-4567ffedf285 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:06.884082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.902458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b144c29-c343-40f1-b764-94a3eaa416f1 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.916777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e686644-8ce9-4f82-9b99-4567ffedf285 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:06.917096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:06.937556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d9055-6a4c-4f37-8b98-b01b72dcca5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.938157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d9055-6a4c-4f37-8b98-b01b72dcca5b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.938441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d9055-6a4c-4f37-8b98-b01b72dcca5b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.938717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d9055-6a4c-4f37-8b98-b01b72dcca5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.950768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c8a3f-e1d3-4a72-92b5-8378d064191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:06.951423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8a3f-e1d3-4a72-92b5-8378d064191d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:06.951702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8a3f-e1d3-4a72-92b5-8378d064191d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:06.951978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8a3f-e1d3-4a72-92b5-8378d064191d None admin] PATH_INFO: `/v3/projects/69a97ec1fee7433aae3ef0397c97c909` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:06.953094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c8a3f-e1d3-4a72-92b5-8378d064191d None admin] RBAC: Authorizing `identity:get_project(project_id=69a97ec1fee7433aae3ef0397c97c909)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:06.954528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c8a3f-e1d3-4a72-92b5-8378d064191d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:06.956354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 547/1099] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/projects/69a97ec1fee7433aae3ef0397c97c909 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:06.962179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c8a3f-e1d3-4a72-92b5-8378d064191d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:07.005968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d9055-6a4c-4f37-8b98-b01b72dcca5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:07.007455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d9055-6a4c-4f37-8b98-b01b72dcca5b None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:07.059775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c8a3f-e1d3-4a72-92b5-8378d064191d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:07.093282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:07.093923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:07.094169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:07.094440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:07.095185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=69a97ec1fee7433aae3ef0397c97c909, user.id=7e47b469355e48d3b37729c8e3d84bb0) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:07.095492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:07.096888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:07.099673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:07.102914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 548/1100] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/role_assignments?user.id=7e47b469355e48d3b37729c8e3d84bb0&scope.project.id=69a97ec1fee7433aae3ef0397c97c909&role.id=f72d3388f13149b483355331380f040b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:05:07.104478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 553/1101] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 12:05:07.109661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:07.219067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d14566-0fd5-4e66-91f3-913bfa0a43bf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:07.255862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:07.256500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:07.256785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:07.257060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:07.272532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:07.274027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:05:07.289603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:07.290620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:05:07.291591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:05:07.291591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:05:07.291591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:07.291591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 549/1102] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:05:07.296642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d9055-6a4c-4f37-8b98-b01b72dcca5b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:07.312479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:07.396403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d9055-6a4c-4f37-8b98-b01b72dcca5b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:07.435490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:07.436090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:07.436364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:07.436631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:07.437266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:07.437563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:07.438925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:07.450305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 554/1103] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:07.455873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:07.508960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:07.554774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fa686-2ce4-4e9d-b850-ad0d7e00bf58 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:07.588080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:07.588623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:07.588900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:07.589167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:07.595782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:07.597198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:05:07.608873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:07.609944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:07.611476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 550/1104] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:05:07.616091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:07.691106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:07.714977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee728dfa-d415-4367-9dd3-c9c4379fcf2c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:07.749849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc68c4e-2b98-47ca-ac59-34cd744bd301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:07.750512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc68c4e-2b98-47ca-ac59-34cd744bd301 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:07.750720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc68c4e-2b98-47ca-ac59-34cd744bd301 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:07.750993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc68c4e-2b98-47ca-ac59-34cd744bd301 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:07.751709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc68c4e-2b98-47ca-ac59-34cd744bd301 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:07.752019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc68c4e-2b98-47ca-ac59-34cd744bd301 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:07.753445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc68c4e-2b98-47ca-ac59-34cd744bd301 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:07.765755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 555/1105] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:07.771760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:07.873179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00381328-16f4-4b78-b4b0-8a6a58fc4cd0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:07.886290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:07.907178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2120038-8c23-4d25-aeae-d0c915cc458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:07.907780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2120038-8c23-4d25-aeae-d0c915cc458a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:07.908063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2120038-8c23-4d25-aeae-d0c915cc458a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:07.908337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2120038-8c23-4d25-aeae-d0c915cc458a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:07.909058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2120038-8c23-4d25-aeae-d0c915cc458a None admin] RBAC: Adding query filter params (role.id=93f3167474ff4071a8c82c01f6fd3f1c, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=cb2075e9fff3459881c13f81456ab198) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:07.909368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2120038-8c23-4d25-aeae-d0c915cc458a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:07.910784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2120038-8c23-4d25-aeae-d0c915cc458a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:07.917272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 551/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:07 2022] GET /identity/v3/role_assignments?user.id=cb2075e9fff3459881c13f81456ab198&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=93f3167474ff4071a8c82c01f6fd3f1c => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:09.133598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c082e8-04f6-4bb5-8f94-07d34d4421cc None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:09.134085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c082e8-04f6-4bb5-8f94-07d34d4421cc None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:09.134182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c082e8-04f6-4bb5-8f94-07d34d4421cc None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:09.135526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 556/1107] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:09.140664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5aca-8283-4903-8a53-3843b9c8480f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:09.141062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5aca-8283-4903-8a53-3843b9c8480f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:09.141221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5aca-8283-4903-8a53-3843b9c8480f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:09.158943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ba8b5aca-8283-4903-8a53-3843b9c8480f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:09.160168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ba8b5aca-8283-4903-8a53-3843b9c8480f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:09.225991 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 552/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:09.251301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c082e8-04f6-4bb5-8f94-07d34d4421cc None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:09.350865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c082e8-04f6-4bb5-8f94-07d34d4421cc None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:09.384771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c039-2153-418a-9a14-5b17525ee79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:09.385387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c039-2153-418a-9a14-5b17525ee79f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:09.385669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c039-2153-418a-9a14-5b17525ee79f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:09.385943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c039-2153-418a-9a14-5b17525ee79f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:09.387077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682c039-2153-418a-9a14-5b17525ee79f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:09.388513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682c039-2153-418a-9a14-5b17525ee79f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:09.390061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 557/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:09.395970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b5aca-8283-4903-8a53-3843b9c8480f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:09.493032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b5aca-8283-4903-8a53-3843b9c8480f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:09.500155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:09.526289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ae8c1-a8e8-4c2d-8409-57b7fd29a9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:09.527024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ae8c1-a8e8-4c2d-8409-57b7fd29a9e6 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:09.527316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ae8c1-a8e8-4c2d-8409-57b7fd29a9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:09.527584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ae8c1-a8e8-4c2d-8409-57b7fd29a9e6 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:09.528213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ae8c1-a8e8-4c2d-8409-57b7fd29a9e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:09.529628 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ae8c1-a8e8-4c2d-8409-57b7fd29a9e6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:09.583543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 553/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:05:09 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:05:11.654131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd94ad4-602b-49db-b884-fff602629bf8 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:11.654694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd94ad4-602b-49db-b884-fff602629bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:11.654694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd94ad4-602b-49db-b884-fff602629bf8 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:11.656032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 558/1111] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:11.661106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f32a-de5d-4516-867e-7d81ee6b05f2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:11.661476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f32a-de5d-4516-867e-7d81ee6b05f2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:11.661653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f32a-de5d-4516-867e-7d81ee6b05f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:11.679222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-70a7f32a-de5d-4516-867e-7d81ee6b05f2 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:11.681078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-70a7f32a-de5d-4516-867e-7d81ee6b05f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:11.745269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 554/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:11.768159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd94ad4-602b-49db-b884-fff602629bf8 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:11.771019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:11.867422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd94ad4-602b-49db-b884-fff602629bf8 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:11.900626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e421c-63bd-4556-8c27-90ee868dc949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:11.901239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e421c-63bd-4556-8c27-90ee868dc949 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:11.901518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e421c-63bd-4556-8c27-90ee868dc949 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:11.901790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e421c-63bd-4556-8c27-90ee868dc949 None admin] PATH_INFO: `/v3/roles/9475450bfdd04912abb88ccf9973f9d8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:11.903196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e421c-63bd-4556-8c27-90ee868dc949 None admin] RBAC: Authorizing `identity:get_role(role_id=9475450bfdd04912abb88ccf9973f9d8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:11.904623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e421c-63bd-4556-8c27-90ee868dc949 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:11.905868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 559/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:11 2022] GET /identity/v3/roles/9475450bfdd04912abb88ccf9973f9d8 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:11.911701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7f32a-de5d-4516-867e-7d81ee6b05f2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:11.999950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:12.008675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7f32a-de5d-4516-867e-7d81ee6b05f2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:12.041988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f110df4-327b-43ab-9e7a-93e29d9b8065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:12.042595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110df4-327b-43ab-9e7a-93e29d9b8065 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:12.042911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110df4-327b-43ab-9e7a-93e29d9b8065 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:12.043207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110df4-327b-43ab-9e7a-93e29d9b8065 None admin] PATH_INFO: `/v3/users/a7c0b8f2240b4b0a9b5445781539f223` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:12.056680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f110df4-327b-43ab-9e7a-93e29d9b8065 None admin] RBAC: Authorizing `identity:get_user(user_id=a7c0b8f2240b4b0a9b5445781539f223)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:12.058118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f110df4-327b-43ab-9e7a-93e29d9b8065 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:12.059760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 555/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:11 2022] GET /identity/v3/users/a7c0b8f2240b4b0a9b5445781539f223 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:12.065358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e421c-63bd-4556-8c27-90ee868dc949 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:12.133946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:12.164686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e421c-63bd-4556-8c27-90ee868dc949 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:12.197616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4769-e055-48fc-9f16-14294b13a098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:12.198216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4769-e055-48fc-9f16-14294b13a098 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:12.198498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4769-e055-48fc-9f16-14294b13a098 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:12.198772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4769-e055-48fc-9f16-14294b13a098 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:12.264071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4769-e055-48fc-9f16-14294b13a098 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:12.265511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4769-e055-48fc-9f16-14294b13a098 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:12.330454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:12.359892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 560/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:05:12.365994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f110df4-327b-43ab-9e7a-93e29d9b8065 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:12.461652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f110df4-327b-43ab-9e7a-93e29d9b8065 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:12.495615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fcf57-2e17-4919-8e28-763d9fb053ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:12.496231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fcf57-2e17-4919-8e28-763d9fb053ae None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:12.496507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fcf57-2e17-4919-8e28-763d9fb053ae None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:12.496773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fcf57-2e17-4919-8e28-763d9fb053ae None admin] PATH_INFO: `/v3/projects/69a97ec1fee7433aae3ef0397c97c909` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:12.497877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57fcf57-2e17-4919-8e28-763d9fb053ae None admin] RBAC: Authorizing `identity:get_project(project_id=69a97ec1fee7433aae3ef0397c97c909)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:12.499298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57fcf57-2e17-4919-8e28-763d9fb053ae None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:12.500911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 556/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:12 2022] GET /identity/v3/projects/69a97ec1fee7433aae3ef0397c97c909 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:12.507082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4769-e055-48fc-9f16-14294b13a098 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:12.604230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4769-e055-48fc-9f16-14294b13a098 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:12.637575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d71528-91aa-401d-bd7e-1f31c209603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:12.638192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d71528-91aa-401d-bd7e-1f31c209603d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:12.638476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d71528-91aa-401d-bd7e-1f31c209603d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:12.638744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d71528-91aa-401d-bd7e-1f31c209603d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:12.639490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d71528-91aa-401d-bd7e-1f31c209603d None admin] RBAC: Adding query filter params (role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=69a97ec1fee7433aae3ef0397c97c909, user.id=a7c0b8f2240b4b0a9b5445781539f223) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:12.639796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d71528-91aa-401d-bd7e-1f31c209603d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:12.641307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d71528-91aa-401d-bd7e-1f31c209603d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:12.646766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 561/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:12 2022] GET /identity/v3/role_assignments?user.id=a7c0b8f2240b4b0a9b5445781539f223&scope.project.id=69a97ec1fee7433aae3ef0397c97c909&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:14.616552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5be87d-e97d-40d3-ac8d-4fa1ca739900 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:14.617594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5be87d-e97d-40d3-ac8d-4fa1ca739900 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:14.617696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5be87d-e97d-40d3-ac8d-4fa1ca739900 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:14.619079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 557/1118] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:14.624115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b90d0-94ef-4a39-a163-c50ce2edab0c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:14.624528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b90d0-94ef-4a39-a163-c50ce2edab0c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:14.624669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b90d0-94ef-4a39-a163-c50ce2edab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:14.642948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1a6b90d0-94ef-4a39-a163-c50ce2edab0c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:14.644165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1a6b90d0-94ef-4a39-a163-c50ce2edab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:14.669753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:14.709551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 562/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:14.731668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5be87d-e97d-40d3-ac8d-4fa1ca739900 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:14.750890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:14.830980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5be87d-e97d-40d3-ac8d-4fa1ca739900 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:14.863416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad090b4-4c0c-4bfa-aadd-a518efc96e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:14.864022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad090b4-4c0c-4bfa-aadd-a518efc96e41 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:14.864297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad090b4-4c0c-4bfa-aadd-a518efc96e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:14.864567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad090b4-4c0c-4bfa-aadd-a518efc96e41 None admin] PATH_INFO: `/v3/roles/9475450bfdd04912abb88ccf9973f9d8` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:14.866188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad090b4-4c0c-4bfa-aadd-a518efc96e41 None admin] RBAC: Authorizing `identity:get_role(role_id=9475450bfdd04912abb88ccf9973f9d8)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:14.868252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad090b4-4c0c-4bfa-aadd-a518efc96e41 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:14.869466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 558/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:14 2022] GET /identity/v3/roles/9475450bfdd04912abb88ccf9973f9d8 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:14.875059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b90d0-94ef-4a39-a163-c50ce2edab0c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:14.969051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b90d0-94ef-4a39-a163-c50ce2edab0c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:15.002520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944215a-0172-4829-953e-cc56b41ba290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:15.003167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944215a-0172-4829-953e-cc56b41ba290 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:15.003447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944215a-0172-4829-953e-cc56b41ba290 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:15.003716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944215a-0172-4829-953e-cc56b41ba290 None admin] PATH_INFO: `/v3/users/a7c0b8f2240b4b0a9b5445781539f223` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:15.004841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944215a-0172-4829-953e-cc56b41ba290 None admin] RBAC: Authorizing `identity:get_user(user_id=a7c0b8f2240b4b0a9b5445781539f223)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:15.006263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944215a-0172-4829-953e-cc56b41ba290 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:15.007920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 563/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:14 2022] GET /identity/v3/users/a7c0b8f2240b4b0a9b5445781539f223 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:15.013474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad090b4-4c0c-4bfa-aadd-a518efc96e41 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:15.095196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:15.110316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad090b4-4c0c-4bfa-aadd-a518efc96e41 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:15.145183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018fade-6ebc-4360-9bf7-d300f5673e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:15.145800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018fade-6ebc-4360-9bf7-d300f5673e24 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:15.146075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018fade-6ebc-4360-9bf7-d300f5673e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:15.146342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018fade-6ebc-4360-9bf7-d300f5673e24 None admin] PATH_INFO: `/v3/projects/69a97ec1fee7433aae3ef0397c97c909` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:15.147475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018fade-6ebc-4360-9bf7-d300f5673e24 None admin] RBAC: Authorizing `identity:get_project(project_id=69a97ec1fee7433aae3ef0397c97c909)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:15.149007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018fade-6ebc-4360-9bf7-d300f5673e24 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:15.150597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 559/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:15 2022] GET /identity/v3/projects/69a97ec1fee7433aae3ef0397c97c909 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:15.156578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944215a-0172-4829-953e-cc56b41ba290 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:15.186745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:15.252504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944215a-0172-4829-953e-cc56b41ba290 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:15.284813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db119b2-084e-4be4-9a4b-66cf3f30dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:15.285447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db119b2-084e-4be4-9a4b-66cf3f30dbb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:15.285724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db119b2-084e-4be4-9a4b-66cf3f30dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:15.285995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db119b2-084e-4be4-9a4b-66cf3f30dbb0 None admin] PATH_INFO: `/v3/projects/69a97ec1fee7433aae3ef0397c97c909/users/a7c0b8f2240b4b0a9b5445781539f223/roles/9475450bfdd04912abb88ccf9973f9d8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:15.288109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db119b2-084e-4be4-9a4b-66cf3f30dbb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=69a97ec1fee7433aae3ef0397c97c909, user_id=a7c0b8f2240b4b0a9b5445781539f223, role_id=9475450bfdd04912abb88ccf9973f9d8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:15.289580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db119b2-084e-4be4-9a4b-66cf3f30dbb0 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:15.381175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 564/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:05:15 2022] PUT /identity/v3/projects/69a97ec1fee7433aae3ef0397c97c909/users/a7c0b8f2240b4b0a9b5445781539f223/roles/9475450bfdd04912abb88ccf9973f9d8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:05:17.333581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127c689-43d7-4743-896e-444b91f32dcb None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:17.334124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127c689-43d7-4743-896e-444b91f32dcb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:17.334124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127c689-43d7-4743-896e-444b91f32dcb None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:17.335484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 560/1124] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:17.341211 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18c7e8-dce1-4874-909b-4584e2edf5d1 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:17.341636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18c7e8-dce1-4874-909b-4584e2edf5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:17.341761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18c7e8-dce1-4874-909b-4584e2edf5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:17.359633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-de18c7e8-dce1-4874-909b-4584e2edf5d1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:17.360757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-de18c7e8-dce1-4874-909b-4584e2edf5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:17.425689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 565/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:17.448694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127c689-43d7-4743-896e-444b91f32dcb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:17.545484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127c689-43d7-4743-896e-444b91f32dcb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:17.578602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23105-dca8-479f-a76e-06bc7a17962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:17.579239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23105-dca8-479f-a76e-06bc7a17962a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:17.579514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23105-dca8-479f-a76e-06bc7a17962a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:17.579780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23105-dca8-479f-a76e-06bc7a17962a None admin] PATH_INFO: `/v3/roles/9475450bfdd04912abb88ccf9973f9d8` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:17.581132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a23105-dca8-479f-a76e-06bc7a17962a None admin] RBAC: Authorizing `identity:get_role(role_id=9475450bfdd04912abb88ccf9973f9d8)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:17.582480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a23105-dca8-479f-a76e-06bc7a17962a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:17.583806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 561/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:17 2022] GET /identity/v3/roles/9475450bfdd04912abb88ccf9973f9d8 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:17.589740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18c7e8-dce1-4874-909b-4584e2edf5d1 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:17.613105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:17.687957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18c7e8-dce1-4874-909b-4584e2edf5d1 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:17.721178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5e948-79b2-4411-99c7-2a74b2179f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:17.721795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e948-79b2-4411-99c7-2a74b2179f5f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:17.722073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e948-79b2-4411-99c7-2a74b2179f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:17.722345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e948-79b2-4411-99c7-2a74b2179f5f None admin] PATH_INFO: `/v3/users/a7c0b8f2240b4b0a9b5445781539f223` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:17.723478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5e948-79b2-4411-99c7-2a74b2179f5f None admin] RBAC: Authorizing `identity:get_user(user_id=a7c0b8f2240b4b0a9b5445781539f223)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:17.724913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5e948-79b2-4411-99c7-2a74b2179f5f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:17.726660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 566/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:17 2022] GET /identity/v3/users/a7c0b8f2240b4b0a9b5445781539f223 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:17.732413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23105-dca8-479f-a76e-06bc7a17962a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:17.757330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:17.832241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23105-dca8-479f-a76e-06bc7a17962a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:17.865347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ffa0b-c1ee-44d3-a532-a720924e6780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:17.866151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ffa0b-c1ee-44d3-a532-a720924e6780 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:17.866430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ffa0b-c1ee-44d3-a532-a720924e6780 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:17.866704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ffa0b-c1ee-44d3-a532-a720924e6780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:17.932468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ffa0b-c1ee-44d3-a532-a720924e6780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:17.933907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ffa0b-c1ee-44d3-a532-a720924e6780 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:17.956592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:18.031504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 562/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:05:18.037583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5e948-79b2-4411-99c7-2a74b2179f5f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:18.124202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:18.133834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5e948-79b2-4411-99c7-2a74b2179f5f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:18.166439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfaf75-32c6-438c-a873-11da2a3a42c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:18.167068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfaf75-32c6-438c-a873-11da2a3a42c7 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:18.167349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfaf75-32c6-438c-a873-11da2a3a42c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:18.167621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfaf75-32c6-438c-a873-11da2a3a42c7 None admin] PATH_INFO: `/v3/projects/69a97ec1fee7433aae3ef0397c97c909` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:18.168714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfaf75-32c6-438c-a873-11da2a3a42c7 None admin] RBAC: Authorizing `identity:get_project(project_id=69a97ec1fee7433aae3ef0397c97c909)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:18.170130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfaf75-32c6-438c-a873-11da2a3a42c7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:18.171758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 567/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:18 2022] GET /identity/v3/projects/69a97ec1fee7433aae3ef0397c97c909 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:18.178019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ffa0b-c1ee-44d3-a532-a720924e6780 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:18.274972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ffa0b-c1ee-44d3-a532-a720924e6780 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:18.296406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:18.308102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc762bb-dbde-46cf-a40b-24c2da2c3301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:18.308696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc762bb-dbde-46cf-a40b-24c2da2c3301 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:18.316591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc762bb-dbde-46cf-a40b-24c2da2c3301 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:18.316591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc762bb-dbde-46cf-a40b-24c2da2c3301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:18.316591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc762bb-dbde-46cf-a40b-24c2da2c3301 None admin] RBAC: Adding query filter params (role.id=9475450bfdd04912abb88ccf9973f9d8, scope.project.id=69a97ec1fee7433aae3ef0397c97c909, user.id=a7c0b8f2240b4b0a9b5445781539f223) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:18.316591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc762bb-dbde-46cf-a40b-24c2da2c3301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:18.316591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc762bb-dbde-46cf-a40b-24c2da2c3301 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:18.317295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 563/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:18 2022] GET /identity/v3/role_assignments?user.id=a7c0b8f2240b4b0a9b5445781539f223&scope.project.id=69a97ec1fee7433aae3ef0397c97c909&role.id=9475450bfdd04912abb88ccf9973f9d8 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:20.302402 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489c48-214d-4d13-a8bc-bd15d4e90865 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:20.302946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489c48-214d-4d13-a8bc-bd15d4e90865 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:20.302946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489c48-214d-4d13-a8bc-bd15d4e90865 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:20.304297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 568/1131] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:20.309382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff208-f037-41f2-983e-d7ad6bf1496c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:20.309785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff208-f037-41f2-983e-d7ad6bf1496c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:20.309937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff208-f037-41f2-983e-d7ad6bf1496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:20.328154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-833ff208-f037-41f2-983e-d7ad6bf1496c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:20.329334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-833ff208-f037-41f2-983e-d7ad6bf1496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:20.393314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 564/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:05:20.416158 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b489c48-214d-4d13-a8bc-bd15d4e90865 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:20.514578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b489c48-214d-4d13-a8bc-bd15d4e90865 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:20.546223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7533f84-30cf-482a-991a-f469342b0ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:20.546825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7533f84-30cf-482a-991a-f469342b0ceb None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:20.547130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7533f84-30cf-482a-991a-f469342b0ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:20.547398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7533f84-30cf-482a-991a-f469342b0ceb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:20.548482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7533f84-30cf-482a-991a-f469342b0ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:20.549896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7533f84-30cf-482a-991a-f469342b0ceb None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:20.551446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 569/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:20.557713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ff208-f037-41f2-983e-d7ad6bf1496c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:20.655124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ff208-f037-41f2-983e-d7ad6bf1496c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:20.670595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:20.687696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a24955-c8f1-4519-bcfb-e7eb67ac9bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:20.693961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a24955-c8f1-4519-bcfb-e7eb67ac9bec None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:20.693961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a24955-c8f1-4519-bcfb-e7eb67ac9bec None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:20.693961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a24955-c8f1-4519-bcfb-e7eb67ac9bec None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:20.693961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a24955-c8f1-4519-bcfb-e7eb67ac9bec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:20.693961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a24955-c8f1-4519-bcfb-e7eb67ac9bec None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:20.806003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 565/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:05:20 2022] POST /identity/v3/projects => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:05:22.846322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8f547-7aa9-4b94-8e4b-c59eb81ea91f None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:22.846920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8f547-7aa9-4b94-8e4b-c59eb81ea91f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:22.846920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8f547-7aa9-4b94-8e4b-c59eb81ea91f None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:22.848221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 570/1135] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:22.853639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb03d1d-841c-4d4b-98eb-87e7f073df36 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:22.854091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb03d1d-841c-4d4b-98eb-87e7f073df36 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:22.854202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb03d1d-841c-4d4b-98eb-87e7f073df36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:22.872726 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1fb03d1d-841c-4d4b-98eb-87e7f073df36 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:22.873905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1fb03d1d-841c-4d4b-98eb-87e7f073df36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:22.938890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 566/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:22.963769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8f547-7aa9-4b94-8e4b-c59eb81ea91f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:22.990269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:23.062777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8f547-7aa9-4b94-8e4b-c59eb81ea91f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:23.096819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461f27c-4dcd-4603-8b2e-6acc94dacead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:23.097371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f27c-4dcd-4603-8b2e-6acc94dacead None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:23.097648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f27c-4dcd-4603-8b2e-6acc94dacead None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:23.097921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f27c-4dcd-4603-8b2e-6acc94dacead None admin] PATH_INFO: `/v3/domains/default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:23.099042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461f27c-4dcd-4603-8b2e-6acc94dacead None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:23.100475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461f27c-4dcd-4603-8b2e-6acc94dacead None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:23.102614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 571/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:05:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:23.108333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb03d1d-841c-4d4b-98eb-87e7f073df36 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:23.201362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:23.205106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb03d1d-841c-4d4b-98eb-87e7f073df36 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:23.237927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd05e3b-ab6a-4da6-9a81-eb0fcec6df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:23.238520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05e3b-ab6a-4da6-9a81-eb0fcec6df1c None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:23.238800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05e3b-ab6a-4da6-9a81-eb0fcec6df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:23.239102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05e3b-ab6a-4da6-9a81-eb0fcec6df1c None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:23.239745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd05e3b-ab6a-4da6-9a81-eb0fcec6df1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:23.241436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd05e3b-ab6a-4da6-9a81-eb0fcec6df1c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:23.324955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 567/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:05:23 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:05:25.361577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a9228-a76a-49c7-8293-24235622767e None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:25.362097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a9228-a76a-49c7-8293-24235622767e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:25.362097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a9228-a76a-49c7-8293-24235622767e None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:25.363446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 572/1139] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:25.403329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab5b53-3997-4ff8-a29a-c135ce00584a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:25.403329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab5b53-3997-4ff8-a29a-c135ce00584a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:25.403329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab5b53-3997-4ff8-a29a-c135ce00584a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:25.403329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-01ab5b53-3997-4ff8-a29a-c135ce00584a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:25.403329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-01ab5b53-3997-4ff8-a29a-c135ce00584a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:25.462514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 568/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:25.485335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a9228-a76a-49c7-8293-24235622767e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:25.577318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:25.584507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a9228-a76a-49c7-8293-24235622767e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:25.618029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8d251-13f3-4ee4-9e9e-fff57d45189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:25.618561 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d251-13f3-4ee4-9e9e-fff57d45189b None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:25.618843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d251-13f3-4ee4-9e9e-fff57d45189b None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:25.619147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d251-13f3-4ee4-9e9e-fff57d45189b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:25.628918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8d251-13f3-4ee4-9e9e-fff57d45189b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-4cf8d251-13f3-4ee4-9e9e-fff57d45189b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:25.634436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:25.635639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:25.637000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 573/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:05:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:25.641565 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab5b53-3997-4ff8-a29a-c135ce00584a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:25.722854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:25.737851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab5b53-3997-4ff8-a29a-c135ce00584a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:25.772152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:25.772748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:25.773031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:25.773372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:25.774191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:25.774528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:25.775916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:25.783842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 569/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:05:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:25.789865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8d251-13f3-4ee4-9e9e-fff57d45189b None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:25.887143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8d251-13f3-4ee4-9e9e-fff57d45189b None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:25.919342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acaf69d-7964-4e5b-b91a-b3fa861f35e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:25.919947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acaf69d-7964-4e5b-b91a-b3fa861f35e2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:25.920230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acaf69d-7964-4e5b-b91a-b3fa861f35e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:25.920506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acaf69d-7964-4e5b-b91a-b3fa861f35e2 None admin] PATH_INFO: `/v3/users/6cf72c2404174868ab7788e3afa44564` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:25.934376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acaf69d-7964-4e5b-b91a-b3fa861f35e2 None admin] RBAC: Authorizing `identity:get_user(user_id=6cf72c2404174868ab7788e3afa44564)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:25.935844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acaf69d-7964-4e5b-b91a-b3fa861f35e2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:25.937408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 574/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:25 2022] GET /identity/v3/users/6cf72c2404174868ab7788e3afa44564 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:25.943035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:26.041513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25d998-1ef5-4d1e-9c2f-a43927b2f325 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:26.079124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337439-1cf2-4582-8f0d-b3d88350f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:26.079722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337439-1cf2-4582-8f0d-b3d88350f899 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:26.080002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337439-1cf2-4582-8f0d-b3d88350f899 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:26.080276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337439-1cf2-4582-8f0d-b3d88350f899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:26.095151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:26.147326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55337439-1cf2-4582-8f0d-b3d88350f899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:26.148884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55337439-1cf2-4582-8f0d-b3d88350f899 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:26.243113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 570/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:05:26.248898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acaf69d-7964-4e5b-b91a-b3fa861f35e2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:26.263306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:26.345661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acaf69d-7964-4e5b-b91a-b3fa861f35e2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:26.379080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb9a9-22e7-4fdf-ac75-51319d5c4c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:26.379697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb9a9-22e7-4fdf-ac75-51319d5c4c1f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:26.379980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb9a9-22e7-4fdf-ac75-51319d5c4c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:26.380255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb9a9-22e7-4fdf-ac75-51319d5c4c1f None admin] PATH_INFO: `/v3/projects/3e428c0405c24a929dec75e0d53e3c5e` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:26.381364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb9a9-22e7-4fdf-ac75-51319d5c4c1f None admin] RBAC: Authorizing `identity:get_project(project_id=3e428c0405c24a929dec75e0d53e3c5e)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:26.383345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb9a9-22e7-4fdf-ac75-51319d5c4c1f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:26.384958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 575/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:26 2022] GET /identity/v3/projects/3e428c0405c24a929dec75e0d53e3c5e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:26.390959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337439-1cf2-4582-8f0d-b3d88350f899 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:26.435744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:26.487082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337439-1cf2-4582-8f0d-b3d88350f899 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:26.519765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68837aae-4b94-4fae-8a97-d8a28097884e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:26.520346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68837aae-4b94-4fae-8a97-d8a28097884e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:26.520625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68837aae-4b94-4fae-8a97-d8a28097884e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:26.520897 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68837aae-4b94-4fae-8a97-d8a28097884e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:26.521605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68837aae-4b94-4fae-8a97-d8a28097884e None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=3e428c0405c24a929dec75e0d53e3c5e, user.id=6cf72c2404174868ab7788e3afa44564) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:26.521910 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68837aae-4b94-4fae-8a97-d8a28097884e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:26.524025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68837aae-4b94-4fae-8a97-d8a28097884e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:26.529141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 571/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:26 2022] GET /identity/v3/role_assignments?user.id=6cf72c2404174868ab7788e3afa44564&scope.project.id=3e428c0405c24a929dec75e0d53e3c5e&role.id=f72d3388f13149b483355331380f040b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:28.489221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d3af3-e7af-4f06-b237-0a2a8c5d357f None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:28.489783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d3af3-e7af-4f06-b237-0a2a8c5d357f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:28.489783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d3af3-e7af-4f06-b237-0a2a8c5d357f None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:28.491403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 576/1147] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:28.496410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbd650-30fb-475d-85b2-4e309e6b5864 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:28.496797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbd650-30fb-475d-85b2-4e309e6b5864 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:28.496963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbd650-30fb-475d-85b2-4e309e6b5864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:28.515322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-36cbd650-30fb-475d-85b2-4e309e6b5864 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:28.516512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-36cbd650-30fb-475d-85b2-4e309e6b5864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:28.580995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 572/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:28.603496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d3af3-e7af-4f06-b237-0a2a8c5d357f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:28.682188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:28.701259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d3af3-e7af-4f06-b237-0a2a8c5d357f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:28.734496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c4a93-4572-4a5e-9ed6-f61600c3406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:28.735143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4a93-4572-4a5e-9ed6-f61600c3406f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:28.735423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4a93-4572-4a5e-9ed6-f61600c3406f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:28.735696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4a93-4572-4a5e-9ed6-f61600c3406f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:28.746554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c4a93-4572-4a5e-9ed6-f61600c3406f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-e11c4a93-4572-4a5e-9ed6-f61600c3406f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:28.751225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:28.752368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:28.753436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 577/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:05:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:28.759137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbd650-30fb-475d-85b2-4e309e6b5864 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:28.798761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:28.855151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbd650-30fb-475d-85b2-4e309e6b5864 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:28.888664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:28.889253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:28.889536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:28.889809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:28.890463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:28.890765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:28.892144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:28.900150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 573/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:05:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:28.905929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c4a93-4572-4a5e-9ed6-f61600c3406f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:29.003926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c4a93-4572-4a5e-9ed6-f61600c3406f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:29.036256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4e5b9-3318-4f40-b534-29e2c39e086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:29.036865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e5b9-3318-4f40-b534-29e2c39e086e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:29.037145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e5b9-3318-4f40-b534-29e2c39e086e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:29.037418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e5b9-3318-4f40-b534-29e2c39e086e None admin] PATH_INFO: `/v3/users/6cf72c2404174868ab7788e3afa44564` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:29.038532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e5b9-3318-4f40-b534-29e2c39e086e None admin] RBAC: Authorizing `identity:get_user(user_id=6cf72c2404174868ab7788e3afa44564)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:29.040367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e5b9-3318-4f40-b534-29e2c39e086e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:29.041930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 578/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:28 2022] GET /identity/v3/users/6cf72c2404174868ab7788e3afa44564 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:29.047533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:29.148826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c5d4-3f5b-4a51-a293-2449f895c59d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:29.150485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:29.182631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8206c-66b6-460e-a2be-1a9e3eb7d9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:29.183275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8206c-66b6-460e-a2be-1a9e3eb7d9bc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:29.183551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8206c-66b6-460e-a2be-1a9e3eb7d9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:29.183816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8206c-66b6-460e-a2be-1a9e3eb7d9bc None admin] PATH_INFO: `/v3/projects/3e428c0405c24a929dec75e0d53e3c5e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:29.185016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8206c-66b6-460e-a2be-1a9e3eb7d9bc None admin] RBAC: Authorizing `identity:get_project(project_id=3e428c0405c24a929dec75e0d53e3c5e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:29.186420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8206c-66b6-460e-a2be-1a9e3eb7d9bc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:29.188046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 574/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:29 2022] GET /identity/v3/projects/3e428c0405c24a929dec75e0d53e3c5e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:29.193962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4e5b9-3318-4f40-b534-29e2c39e086e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:29.205888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:29.292313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4e5b9-3318-4f40-b534-29e2c39e086e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:29.324947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8de8d4-40dc-498a-b686-566ae55999da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:29.325582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8de8d4-40dc-498a-b686-566ae55999da None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:29.325858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8de8d4-40dc-498a-b686-566ae55999da None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:29.326126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8de8d4-40dc-498a-b686-566ae55999da None admin] PATH_INFO: `/v3/projects/3e428c0405c24a929dec75e0d53e3c5e/users/6cf72c2404174868ab7788e3afa44564/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:29.328011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8de8d4-40dc-498a-b686-566ae55999da None admin] RBAC: Authorizing `identity:create_grant(project_id=3e428c0405c24a929dec75e0d53e3c5e, user_id=6cf72c2404174868ab7788e3afa44564, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:29.329428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8de8d4-40dc-498a-b686-566ae55999da None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:29.389333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 579/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:05:29 2022] PUT /identity/v3/projects/3e428c0405c24a929dec75e0d53e3c5e/users/6cf72c2404174868ab7788e3afa44564/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:05:31.412832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382983-01a0-4e65-85cb-097e6ae677ef None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:31.413403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382983-01a0-4e65-85cb-097e6ae677ef None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:31.413403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382983-01a0-4e65-85cb-097e6ae677ef None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:31.415497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 575/1154] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:31.420558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7fdeb-e452-43d5-a0f8-89449b00a7c4 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:31.421000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7fdeb-e452-43d5-a0f8-89449b00a7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:31.421103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7fdeb-e452-43d5-a0f8-89449b00a7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:31.439712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-dcb7fdeb-e452-43d5-a0f8-89449b00a7c4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:31.440894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-dcb7fdeb-e452-43d5-a0f8-89449b00a7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:31.504827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:31.510576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 580/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:05:31.534100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84382983-01a0-4e65-85cb-097e6ae677ef None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:31.636417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84382983-01a0-4e65-85cb-097e6ae677ef None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:31.669855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbf24b-5c43-4eac-8ab7-37749f7e8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:31.670403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf24b-5c43-4eac-8ab7-37749f7e8a0f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:31.670675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf24b-5c43-4eac-8ab7-37749f7e8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:31.670939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf24b-5c43-4eac-8ab7-37749f7e8a0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:31.681303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbf24b-5c43-4eac-8ab7-37749f7e8a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c4bbf24b-5c43-4eac-8ab7-37749f7e8a0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:31.685886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:31.686979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:31.688023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 576/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:05:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:31.692882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7fdeb-e452-43d5-a0f8-89449b00a7c4 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:31.789551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7fdeb-e452-43d5-a0f8-89449b00a7c4 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:31.822329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:31.822912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:31.823212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:31.823477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:31.824131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:31.824429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:31.825785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:31.833513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 581/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:05:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:31.840217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbf24b-5c43-4eac-8ab7-37749f7e8a0f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:31.871691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:31.940713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbf24b-5c43-4eac-8ab7-37749f7e8a0f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:31.983224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d2e5f-ef5c-4d5d-8967-a84e8de5621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:31.983840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d2e5f-ef5c-4d5d-8967-a84e8de5621c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:31.984120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d2e5f-ef5c-4d5d-8967-a84e8de5621c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:31.984392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d2e5f-ef5c-4d5d-8967-a84e8de5621c None admin] PATH_INFO: `/v3/users/6cf72c2404174868ab7788e3afa44564` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:31.985505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d2e5f-ef5c-4d5d-8967-a84e8de5621c None admin] RBAC: Authorizing `identity:get_user(user_id=6cf72c2404174868ab7788e3afa44564)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:31.986921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d2e5f-ef5c-4d5d-8967-a84e8de5621c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:31.988503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 577/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:31 2022] GET /identity/v3/users/6cf72c2404174868ab7788e3afa44564 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:31.994990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:32.052138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:32.093456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf69e4e-5278-419d-9d98-52250aceb39c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:32.126822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4860b-fad9-4e69-9b5c-af6d2de22cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:32.127446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4860b-fad9-4e69-9b5c-af6d2de22cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:32.127724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4860b-fad9-4e69-9b5c-af6d2de22cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:32.127993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4860b-fad9-4e69-9b5c-af6d2de22cd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:32.192008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4860b-fad9-4e69-9b5c-af6d2de22cd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:32.193376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4860b-fad9-4e69-9b5c-af6d2de22cd3 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:32.248447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:32.288958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 582/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:05:32.299824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d2e5f-ef5c-4d5d-8967-a84e8de5621c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:32.393371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:32.397103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d2e5f-ef5c-4d5d-8967-a84e8de5621c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:32.431214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed9177-8310-4803-8ccc-477caedbce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:32.431823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9177-8310-4803-8ccc-477caedbce3d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:32.432100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9177-8310-4803-8ccc-477caedbce3d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:32.432374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9177-8310-4803-8ccc-477caedbce3d None admin] PATH_INFO: `/v3/projects/3e428c0405c24a929dec75e0d53e3c5e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:32.433465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed9177-8310-4803-8ccc-477caedbce3d None admin] RBAC: Authorizing `identity:get_project(project_id=3e428c0405c24a929dec75e0d53e3c5e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:32.434873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed9177-8310-4803-8ccc-477caedbce3d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:32.436560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 578/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:32 2022] GET /identity/v3/projects/3e428c0405c24a929dec75e0d53e3c5e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:32.442615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4860b-fad9-4e69-9b5c-af6d2de22cd3 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:32.539325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4860b-fad9-4e69-9b5c-af6d2de22cd3 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:32.572593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd50468-83af-4864-92a3-993a32952c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:32.573170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd50468-83af-4864-92a3-993a32952c78 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:32.573445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd50468-83af-4864-92a3-993a32952c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:32.573715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd50468-83af-4864-92a3-993a32952c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:32.574432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd50468-83af-4864-92a3-993a32952c78 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=3e428c0405c24a929dec75e0d53e3c5e, user.id=6cf72c2404174868ab7788e3afa44564) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:32.574737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd50468-83af-4864-92a3-993a32952c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:32.576190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd50468-83af-4864-92a3-993a32952c78 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:32.581672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 583/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:32 2022] GET /identity/v3/role_assignments?user.id=6cf72c2404174868ab7788e3afa44564&scope.project.id=3e428c0405c24a929dec75e0d53e3c5e&role.id=f72d3388f13149b483355331380f040b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:34.598018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb40e-003f-4759-b93b-3a4ef6e7e2dd None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:34.598546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb40e-003f-4759-b93b-3a4ef6e7e2dd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:34.598633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb40e-003f-4759-b93b-3a4ef6e7e2dd None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:34.599998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 579/1162] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:34.605743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0144a-7643-4c30-b618-0b1d7ca8c1bb None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:34.606176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0144a-7643-4c30-b618-0b1d7ca8c1bb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:34.606272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0144a-7643-4c30-b618-0b1d7ca8c1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:34.624502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0fe0144a-7643-4c30-b618-0b1d7ca8c1bb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:34.625666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0fe0144a-7643-4c30-b618-0b1d7ca8c1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:34.642278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:34.691460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 584/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:34.712356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb40e-003f-4759-b93b-3a4ef6e7e2dd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:34.809440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb40e-003f-4759-b93b-3a4ef6e7e2dd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:34.841923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db6738-40bb-4b4e-938e-7f40e30271bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:34.842509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db6738-40bb-4b4e-938e-7f40e30271bb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:34.842783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db6738-40bb-4b4e-938e-7f40e30271bb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:34.843075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db6738-40bb-4b4e-938e-7f40e30271bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:34.875516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:34.908537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db6738-40bb-4b4e-938e-7f40e30271bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:34.909967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db6738-40bb-4b4e-938e-7f40e30271bb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:35.006497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 580/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:05:35.012774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0144a-7643-4c30-b618-0b1d7ca8c1bb None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:35.107428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0144a-7643-4c30-b618-0b1d7ca8c1bb None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:35.140521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46e661-e0a7-4793-832a-bbecbc6f06f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:35.141069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46e661-e0a7-4793-832a-bbecbc6f06f2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:35.141345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46e661-e0a7-4793-832a-bbecbc6f06f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:35.141611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46e661-e0a7-4793-832a-bbecbc6f06f2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:35.148588 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46e661-e0a7-4793-832a-bbecbc6f06f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:35.150167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46e661-e0a7-4793-832a-bbecbc6f06f2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-6d46e661-e0a7-4793-832a-bbecbc6f06f2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:35.161804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:35.163102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:35.164006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 585/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:05:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:35.168941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db6738-40bb-4b4e-938e-7f40e30271bb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:35.231534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:35.265023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db6738-40bb-4b4e-938e-7f40e30271bb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:35.297776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4faf6-f176-4e51-9246-99bf68f1e9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:35.298613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4faf6-f176-4e51-9246-99bf68f1e9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:35.299080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4faf6-f176-4e51-9246-99bf68f1e9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:35.299781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4faf6-f176-4e51-9246-99bf68f1e9a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:35.300433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4faf6-f176-4e51-9246-99bf68f1e9a6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:35.300740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4faf6-f176-4e51-9246-99bf68f1e9a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:35.302101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4faf6-f176-4e51-9246-99bf68f1e9a6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:35.309202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 581/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Sun Aug 7 12:05:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:37.267306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ec29-aa37-4d70-9374-54b52d51783e None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:37.267826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ec29-aa37-4d70-9374-54b52d51783e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:37.267941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ec29-aa37-4d70-9374-54b52d51783e None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:37.269187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 586/1167] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:37.274509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4f0cb-e01b-48fe-b342-ec5e327ad349 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:37.274940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4f0cb-e01b-48fe-b342-ec5e327ad349 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:37.275077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4f0cb-e01b-48fe-b342-ec5e327ad349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:37.292721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d4f4f0cb-e01b-48fe-b342-ec5e327ad349 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:37.293915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d4f4f0cb-e01b-48fe-b342-ec5e327ad349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:37.359135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 582/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:37.382257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ec29-aa37-4d70-9374-54b52d51783e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:37.397714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:37.481399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ec29-aa37-4d70-9374-54b52d51783e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:37.516556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59035055-d59d-48fd-9289-b278a745be5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:37.517148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035055-d59d-48fd-9289-b278a745be5e None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:37.517432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035055-d59d-48fd-9289-b278a745be5e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:37.517709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035055-d59d-48fd-9289-b278a745be5e None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:37.518374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59035055-d59d-48fd-9289-b278a745be5e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:37.519784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59035055-d59d-48fd-9289-b278a745be5e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:37.596440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 587/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Sun Aug 7 12:05:37 2022] POST /identity/v3/domains => generated 267 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:05:39.606004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a350-b2ce-4cc6-9a07-0b75376e6e3b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:39.606535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a350-b2ce-4cc6-9a07-0b75376e6e3b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:39.606535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a350-b2ce-4cc6-9a07-0b75376e6e3b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:39.607933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 583/1170] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:39.613246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbc7c7-57df-4634-836e-c43c603ffa2b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:39.613661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbc7c7-57df-4634-836e-c43c603ffa2b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:39.613789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbc7c7-57df-4634-836e-c43c603ffa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:39.631552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bfbbc7c7-57df-4634-836e-c43c603ffa2b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:39.633287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bfbbc7c7-57df-4634-836e-c43c603ffa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:39.696855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 588/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 12:05:39.719789 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a350-b2ce-4cc6-9a07-0b75376e6e3b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:39.779056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:39.818785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a350-b2ce-4cc6-9a07-0b75376e6e3b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:39.852420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61772f66-e9de-43ef-ac77-c92a9e040fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:39.853031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61772f66-e9de-43ef-ac77-c92a9e040fbf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:39.853311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61772f66-e9de-43ef-ac77-c92a9e040fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:39.853580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61772f66-e9de-43ef-ac77-c92a9e040fbf None admin] PATH_INFO: `/v3/domains/94760804ed3b414b88144bf9ef4d130a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:39.863555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61772f66-e9de-43ef-ac77-c92a9e040fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=94760804ed3b414b88144bf9ef4d130a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:39.864931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61772f66-e9de-43ef-ac77-c92a9e040fbf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:39.867385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 584/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Sun Aug 7 12:05:39 2022] GET /identity/v3/domains/94760804ed3b414b88144bf9ef4d130a => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:39.873252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbc7c7-57df-4634-836e-c43c603ffa2b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:39.876297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:39.968892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbc7c7-57df-4634-836e-c43c603ffa2b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:40.001938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62eeda-07fb-41b1-9b1a-63523fc34bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:40.002522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62eeda-07fb-41b1-9b1a-63523fc34bb7 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:40.002796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62eeda-07fb-41b1-9b1a-63523fc34bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:40.003092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62eeda-07fb-41b1-9b1a-63523fc34bb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:40.003733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62eeda-07fb-41b1-9b1a-63523fc34bb7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:40.005145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62eeda-07fb-41b1-9b1a-63523fc34bb7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:40.088797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 589/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:05:39 2022] POST /identity/v3/projects => generated 361 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:05:42.099469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1320bef-2935-49df-bb2f-2f226f769d2c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:42.099998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1320bef-2935-49df-bb2f-2f226f769d2c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:42.099998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1320bef-2935-49df-bb2f-2f226f769d2c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:42.101383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 585/1174] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:42.106617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a251212-dc6a-4514-9b61-8a8a855a0f91 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:42.107058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a251212-dc6a-4514-9b61-8a8a855a0f91 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:42.107205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a251212-dc6a-4514-9b61-8a8a855a0f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:42.125295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6a251212-dc6a-4514-9b61-8a8a855a0f91 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:42.126460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6a251212-dc6a-4514-9b61-8a8a855a0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:42.174991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:42.190516 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 590/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:42.215919 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1320bef-2935-49df-bb2f-2f226f769d2c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:42.315246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1320bef-2935-49df-bb2f-2f226f769d2c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:42.334753 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:42.347931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a111d98-833b-400d-997a-c8954287dda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:42.348546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a111d98-833b-400d-997a-c8954287dda1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:42.348826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a111d98-833b-400d-997a-c8954287dda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:42.349133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a111d98-833b-400d-997a-c8954287dda1 None admin] PATH_INFO: `/v3/domains/94760804ed3b414b88144bf9ef4d130a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:42.350253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a111d98-833b-400d-997a-c8954287dda1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=94760804ed3b414b88144bf9ef4d130a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:42.351673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a111d98-833b-400d-997a-c8954287dda1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:42.353220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 586/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Sun Aug 7 12:05:42 2022] GET /identity/v3/domains/94760804ed3b414b88144bf9ef4d130a => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:42.358881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a251212-dc6a-4514-9b61-8a8a855a0f91 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:42.455878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a251212-dc6a-4514-9b61-8a8a855a0f91 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:42.487681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bc6f0-1001-4970-9d9f-23dd2bd0a450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:42.488269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bc6f0-1001-4970-9d9f-23dd2bd0a450 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:42.488547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bc6f0-1001-4970-9d9f-23dd2bd0a450 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:42.488813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bc6f0-1001-4970-9d9f-23dd2bd0a450 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:42.489435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bc6f0-1001-4970-9d9f-23dd2bd0a450 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:42.490852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bc6f0-1001-4970-9d9f-23dd2bd0a450 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:42.595353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 591/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:05:42 2022] POST /identity/v3/users => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:05:44.607504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c4c9-5a01-4c6c-9978-aafdac2e1a09 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:44.608042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c4c9-5a01-4c6c-9978-aafdac2e1a09 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:44.608042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c4c9-5a01-4c6c-9978-aafdac2e1a09 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:44.609371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 587/1178] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:44.614366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53b4c1-256a-4490-abf1-d24dd7396e89 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:44.614736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53b4c1-256a-4490-abf1-d24dd7396e89 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:44.614925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53b4c1-256a-4490-abf1-d24dd7396e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:44.633234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6d53b4c1-256a-4490-abf1-d24dd7396e89 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:44.635032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6d53b4c1-256a-4490-abf1-d24dd7396e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:44.668879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:44.701531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 592/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:44.723859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4c4c9-5a01-4c6c-9978-aafdac2e1a09 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:44.826355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4c4c9-5a01-4c6c-9978-aafdac2e1a09 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:44.859105 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba4678-bdc9-435e-a7b3-df7950a7cfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:44.859660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba4678-bdc9-435e-a7b3-df7950a7cfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:44.859941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba4678-bdc9-435e-a7b3-df7950a7cfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:44.860214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba4678-bdc9-435e-a7b3-df7950a7cfd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:44.870334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba4678-bdc9-435e-a7b3-df7950a7cfd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f5ba4678-bdc9-435e-a7b3-df7950a7cfd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:44.874956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:44.876069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:44.877239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 588/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:05:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:44.882009 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53b4c1-256a-4490-abf1-d24dd7396e89 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:44.976652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53b4c1-256a-4490-abf1-d24dd7396e89 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:45.010072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:45.010667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:45.010948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:45.011249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:45.011912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:45.012214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:45.013577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:45.021573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 593/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:05:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:45.027721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba4678-bdc9-435e-a7b3-df7950a7cfd3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:45.076593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:45.127170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba4678-bdc9-435e-a7b3-df7950a7cfd3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:45.161118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04badc-6103-4756-ae32-041dc0b24070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:45.161737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04badc-6103-4756-ae32-041dc0b24070 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:45.162015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04badc-6103-4756-ae32-041dc0b24070 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:45.162285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04badc-6103-4756-ae32-041dc0b24070 None admin] PATH_INFO: `/v3/users/62d0f508096d4a0a9e1427eb96cd19d6` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:45.174890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04badc-6103-4756-ae32-041dc0b24070 None admin] RBAC: Authorizing `identity:get_user(user_id=62d0f508096d4a0a9e1427eb96cd19d6)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:45.176320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04badc-6103-4756-ae32-041dc0b24070 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:45.177904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 589/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:45 2022] GET /identity/v3/users/62d0f508096d4a0a9e1427eb96cd19d6 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:45.184455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:45.218303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:45.282420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fdaa3-b243-411f-95ff-6b29121c8ebd None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:45.314769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110fc0b-7a75-4fb0-bf46-9479f9ee8e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:45.315375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110fc0b-7a75-4fb0-bf46-9479f9ee8e77 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:45.315656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110fc0b-7a75-4fb0-bf46-9479f9ee8e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:45.315928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110fc0b-7a75-4fb0-bf46-9479f9ee8e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:45.380278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110fc0b-7a75-4fb0-bf46-9479f9ee8e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:45.381685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110fc0b-7a75-4fb0-bf46-9479f9ee8e77 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:45.412901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:45.477065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 594/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:05:45.483594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04badc-6103-4756-ae32-041dc0b24070 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:45.579184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04badc-6103-4756-ae32-041dc0b24070 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:45.586163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:45.611545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86310283-83f0-43e5-8547-b7f17ddd52d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:45.612158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86310283-83f0-43e5-8547-b7f17ddd52d5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:45.612430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86310283-83f0-43e5-8547-b7f17ddd52d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:45.612697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86310283-83f0-43e5-8547-b7f17ddd52d5 None admin] PATH_INFO: `/v3/projects/d4567cb4194a4e5196cd5b94308ee7f7` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:45.613789 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86310283-83f0-43e5-8547-b7f17ddd52d5 None admin] RBAC: Authorizing `identity:get_project(project_id=d4567cb4194a4e5196cd5b94308ee7f7)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:45.615276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86310283-83f0-43e5-8547-b7f17ddd52d5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:45.617592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 590/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:45 2022] GET /identity/v3/projects/d4567cb4194a4e5196cd5b94308ee7f7 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:45.623554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110fc0b-7a75-4fb0-bf46-9479f9ee8e77 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:45.718247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110fc0b-7a75-4fb0-bf46-9479f9ee8e77 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:45.751230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed377fc3-97c2-4db4-af88-2831e96dd928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:45.751812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed377fc3-97c2-4db4-af88-2831e96dd928 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:45.752091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed377fc3-97c2-4db4-af88-2831e96dd928 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:45.752359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed377fc3-97c2-4db4-af88-2831e96dd928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:45.753065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed377fc3-97c2-4db4-af88-2831e96dd928 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=d4567cb4194a4e5196cd5b94308ee7f7, user.id=62d0f508096d4a0a9e1427eb96cd19d6) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:45.753367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed377fc3-97c2-4db4-af88-2831e96dd928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:45.754771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed377fc3-97c2-4db4-af88-2831e96dd928 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:45.756460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:45.760728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 595/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:45 2022] GET /identity/v3/role_assignments?user.id=62d0f508096d4a0a9e1427eb96cd19d6&scope.project.id=d4567cb4194a4e5196cd5b94308ee7f7&role.id=f72d3388f13149b483355331380f040b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:47.716366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8d4e-b21b-43d3-b7a7-a2ee0d2eef1a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:47.716900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8d4e-b21b-43d3-b7a7-a2ee0d2eef1a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:47.716900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8d4e-b21b-43d3-b7a7-a2ee0d2eef1a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:47.718255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 591/1186] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:47.723405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d077a3-7b8c-4a16-b90e-4339bb5fd4d9 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:47.723804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d077a3-7b8c-4a16-b90e-4339bb5fd4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:47.723948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d077a3-7b8c-4a16-b90e-4339bb5fd4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:47.741682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f0d077a3-7b8c-4a16-b90e-4339bb5fd4d9 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:47.742846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f0d077a3-7b8c-4a16-b90e-4339bb5fd4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:47.806903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 596/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:47.829459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c8d4e-b21b-43d3-b7a7-a2ee0d2eef1a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:47.928166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c8d4e-b21b-43d3-b7a7-a2ee0d2eef1a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:47.960616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37458e-5b45-41af-ae0a-47366e33e7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:47.961231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37458e-5b45-41af-ae0a-47366e33e7ec None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:47.961509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37458e-5b45-41af-ae0a-47366e33e7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:47.961859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37458e-5b45-41af-ae0a-47366e33e7ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:47.971580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37458e-5b45-41af-ae0a-47366e33e7ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9b37458e-5b45-41af-ae0a-47366e33e7ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:47.976107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:47.977152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:05:47.978654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 592/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:05:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:47.983765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d077a3-7b8c-4a16-b90e-4339bb5fd4d9 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:48.080589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d077a3-7b8c-4a16-b90e-4339bb5fd4d9 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:48.106114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:48.113918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:48.124174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:48.124174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:48.124174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:48.124174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:48.124174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:48.124174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:48.126493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 597/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:05:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:48.132285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37458e-5b45-41af-ae0a-47366e33e7ec None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:48.167201 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:48.232002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37458e-5b45-41af-ae0a-47366e33e7ec None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:48.264170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b7718-4a17-4f59-bc6f-52baeec6452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:48.264781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b7718-4a17-4f59-bc6f-52baeec6452b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:48.265056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b7718-4a17-4f59-bc6f-52baeec6452b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:48.265325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b7718-4a17-4f59-bc6f-52baeec6452b None admin] PATH_INFO: `/v3/users/62d0f508096d4a0a9e1427eb96cd19d6` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:48.266424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b7718-4a17-4f59-bc6f-52baeec6452b None admin] RBAC: Authorizing `identity:get_user(user_id=62d0f508096d4a0a9e1427eb96cd19d6)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:48.267855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b7718-4a17-4f59-bc6f-52baeec6452b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:48.269408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 593/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:48 2022] GET /identity/v3/users/62d0f508096d4a0a9e1427eb96cd19d6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:48.275580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:48.372481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb0372-ee48-48d4-b2e3-0b722ecf1d16 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:48.404845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5f30-6afd-4339-bab2-ddcbadbdbfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:48.405455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5f30-6afd-4339-bab2-ddcbadbdbfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:48.405733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5f30-6afd-4339-bab2-ddcbadbdbfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:48.406004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5f30-6afd-4339-bab2-ddcbadbdbfc4 None admin] PATH_INFO: `/v3/projects/d4567cb4194a4e5196cd5b94308ee7f7` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:48.407136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5f30-6afd-4339-bab2-ddcbadbdbfc4 None admin] RBAC: Authorizing `identity:get_project(project_id=d4567cb4194a4e5196cd5b94308ee7f7)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:48.408562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5f30-6afd-4339-bab2-ddcbadbdbfc4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:48.410153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 598/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:48 2022] GET /identity/v3/projects/d4567cb4194a4e5196cd5b94308ee7f7 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:48.416162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b7718-4a17-4f59-bc6f-52baeec6452b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:48.513237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b7718-4a17-4f59-bc6f-52baeec6452b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:48.514656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:48.547542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedf5c2-a92d-44e2-9c41-35d96313bb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:48.548481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf5c2-a92d-44e2-9c41-35d96313bb1d None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:48.548764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf5c2-a92d-44e2-9c41-35d96313bb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:48.549040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf5c2-a92d-44e2-9c41-35d96313bb1d None admin] PATH_INFO: `/v3/projects/d4567cb4194a4e5196cd5b94308ee7f7/users/62d0f508096d4a0a9e1427eb96cd19d6/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:48.550904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedf5c2-a92d-44e2-9c41-35d96313bb1d None admin] RBAC: Authorizing `identity:create_grant(project_id=d4567cb4194a4e5196cd5b94308ee7f7, user_id=62d0f508096d4a0a9e1427eb96cd19d6, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:48.552336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedf5c2-a92d-44e2-9c41-35d96313bb1d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:48.632861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 594/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:05:48 2022] PUT /identity/v3/projects/d4567cb4194a4e5196cd5b94308ee7f7/users/62d0f508096d4a0a9e1427eb96cd19d6/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:05:50.592618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121713e0-77ab-45b6-8a47-d5374f8e2614 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:50.593175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121713e0-77ab-45b6-8a47-d5374f8e2614 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:50.593175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121713e0-77ab-45b6-8a47-d5374f8e2614 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:50.594528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 599/1193] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:05:50.599695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eea34-0f9b-4ad5-93f6-55abffef7382 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:50.600114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eea34-0f9b-4ad5-93f6-55abffef7382 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:50.600229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eea34-0f9b-4ad5-93f6-55abffef7382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:50.618638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6b9eea34-0f9b-4ad5-93f6-55abffef7382 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:05:50.619921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6b9eea34-0f9b-4ad5-93f6-55abffef7382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:05:50.683579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 595/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:05:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:05:50.706009 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121713e0-77ab-45b6-8a47-d5374f8e2614 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:50.764206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:50.804746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121713e0-77ab-45b6-8a47-d5374f8e2614 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:50.837702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d27b3-ce4e-400e-9449-659afed9d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:50.838302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d27b3-ce4e-400e-9449-659afed9d312 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:50.838581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d27b3-ce4e-400e-9449-659afed9d312 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:50.838852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d27b3-ce4e-400e-9449-659afed9d312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:50.848943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d27b3-ce4e-400e-9449-659afed9d312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-047d27b3-ce4e-400e-9449-659afed9d312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:05:50.853500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:05:50.854583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:05:50.855606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 600/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:05:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:05:50.860576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eea34-0f9b-4ad5-93f6-55abffef7382 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:50.948594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:50.956903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eea34-0f9b-4ad5-93f6-55abffef7382 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:50.990046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:50.990634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:50.990914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:50.991210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:50.992121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:50.992418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:50.993836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:51.001839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 596/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:51.007544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d27b3-ce4e-400e-9449-659afed9d312 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:51.105979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d27b3-ce4e-400e-9449-659afed9d312 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:51.123082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:51.139866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359b8ac-22a9-40c1-873a-afb577126cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:51.140485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b8ac-22a9-40c1-873a-afb577126cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:51.140763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b8ac-22a9-40c1-873a-afb577126cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:51.141035 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b8ac-22a9-40c1-873a-afb577126cc7 None admin] PATH_INFO: `/v3/users/62d0f508096d4a0a9e1427eb96cd19d6` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:51.142143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359b8ac-22a9-40c1-873a-afb577126cc7 None admin] RBAC: Authorizing `identity:get_user(user_id=62d0f508096d4a0a9e1427eb96cd19d6)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:51.143627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359b8ac-22a9-40c1-873a-afb577126cc7 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:51.145193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 601/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Sun Aug 7 12:05:51 2022] GET /identity/v3/users/62d0f508096d4a0a9e1427eb96cd19d6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:51.151712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:51.249578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663e6ba-9d91-427a-be09-1db6c002d344 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:51.282263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c93c-5cfa-41b3-8644-5605100f3e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:51.282851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c93c-5cfa-41b3-8644-5605100f3e87 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:51.283171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c93c-5cfa-41b3-8644-5605100f3e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:51.283434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c93c-5cfa-41b3-8644-5605100f3e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:51.306952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:51.349758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29c93c-5cfa-41b3-8644-5605100f3e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:51.351211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29c93c-5cfa-41b3-8644-5605100f3e87 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:51.444786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 597/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:05:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:05:51.451202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359b8ac-22a9-40c1-873a-afb577126cc7 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:51.547698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359b8ac-22a9-40c1-873a-afb577126cc7 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:51.582042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ebcf1-2030-4c04-83ea-5232b89f716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:51.582653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ebcf1-2030-4c04-83ea-5232b89f716c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:51.582931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ebcf1-2030-4c04-83ea-5232b89f716c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:51.583230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ebcf1-2030-4c04-83ea-5232b89f716c None admin] PATH_INFO: `/v3/projects/d4567cb4194a4e5196cd5b94308ee7f7` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:51.584355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ebcf1-2030-4c04-83ea-5232b89f716c None admin] RBAC: Authorizing `identity:get_project(project_id=d4567cb4194a4e5196cd5b94308ee7f7)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:51.585788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ebcf1-2030-4c04-83ea-5232b89f716c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:51.587518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 602/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Sun Aug 7 12:05:51 2022] GET /identity/v3/projects/d4567cb4194a4e5196cd5b94308ee7f7 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:05:51.593529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c93c-5cfa-41b3-8644-5605100f3e87 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:05:51.646830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:05:51.690799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c93c-5cfa-41b3-8644-5605100f3e87 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:05:51.723699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbfbf6-f24e-40fb-ba56-01d6329b46f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:05:51.724225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbfbf6-f24e-40fb-ba56-01d6329b46f8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:05:51.724505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbfbf6-f24e-40fb-ba56-01d6329b46f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:05:51.724779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbfbf6-f24e-40fb-ba56-01d6329b46f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:05:51.725494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbfbf6-f24e-40fb-ba56-01d6329b46f8 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=d4567cb4194a4e5196cd5b94308ee7f7, user.id=62d0f508096d4a0a9e1427eb96cd19d6) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:05:51.725799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbfbf6-f24e-40fb-ba56-01d6329b46f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:05:51.728037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbfbf6-f24e-40fb-ba56-01d6329b46f8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:05:51.733640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 598/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:05:51 2022] GET /identity/v3/role_assignments?user.id=62d0f508096d4a0a9e1427eb96cd19d6&scope.project.id=d4567cb4194a4e5196cd5b94308ee7f7&role.id=f72d3388f13149b483355331380f040b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:06:01.577722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=73 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:06:01.731084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=41 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:03.267539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ebcd2-b2ca-4400-8cf6-7c3482caa32b None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:03.268125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ebcd2-b2ca-4400-8cf6-7c3482caa32b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:03.268125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ebcd2-b2ca-4400-8cf6-7c3482caa32b None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:03.269439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 603/1201] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:03.274630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebefe7-af4b-4a70-ae61-6e0c57e4a0b5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:03.275048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebefe7-af4b-4a70-ae61-6e0c57e4a0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:03.275211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebefe7-af4b-4a70-ae61-6e0c57e4a0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:03.294220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d6ebefe7-af4b-4a70-ae61-6e0c57e4a0b5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:03.295596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6ebefe7-af4b-4a70-ae61-6e0c57e4a0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:03.362021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 599/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:03.387500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ebcd2-b2ca-4400-8cf6-7c3482caa32b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:03.500879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ebcd2-b2ca-4400-8cf6-7c3482caa32b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:03.534517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:03.535197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:03.535480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:03.535750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:03.546006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:03.547510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:03.561393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:03.562586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:03.563629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:03.563629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:03.563629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:03.563629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 604/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:07:03.568760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebefe7-af4b-4a70-ae61-6e0c57e4a0b5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:03.666186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebefe7-af4b-4a70-ae61-6e0c57e4a0b5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:03.693216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:03.699379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f253b9-e184-4aee-94ca-9bc888d33c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:03.700585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f253b9-e184-4aee-94ca-9bc888d33c3f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:03.700585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f253b9-e184-4aee-94ca-9bc888d33c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:03.700585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f253b9-e184-4aee-94ca-9bc888d33c3f None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:03.701196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f253b9-e184-4aee-94ca-9bc888d33c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:03.701494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f253b9-e184-4aee-94ca-9bc888d33c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:03.702846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f253b9-e184-4aee-94ca-9bc888d33c3f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:03.713600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 600/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:03.720066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:03.773133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:03.817757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d38c56-16c3-4b06-bfc4-73cffc418ac2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:03.851176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b5a29-2643-4dc0-8db3-c3cf59ebcddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:03.851762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b5a29-2643-4dc0-8db3-c3cf59ebcddd None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:03.852037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b5a29-2643-4dc0-8db3-c3cf59ebcddd None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:03.852344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b5a29-2643-4dc0-8db3-c3cf59ebcddd None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:03.852976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b5a29-2643-4dc0-8db3-c3cf59ebcddd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:03.854392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b5a29-2643-4dc0-8db3-c3cf59ebcddd None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:03.956623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 605/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Sun Aug 7 12:07:03 2022] POST /identity/v3/users => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:07:06.005036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c89fa-3ae5-4c3d-b279-4551eecd04ac None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:06.005590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c89fa-3ae5-4c3d-b279-4551eecd04ac None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:06.005590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c89fa-3ae5-4c3d-b279-4551eecd04ac None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:06.006918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 601/1206] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:06.012841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f682cdc-9be2-4c59-b8b4-68d5ed29046c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:06.013259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f682cdc-9be2-4c59-b8b4-68d5ed29046c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:06.013411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f682cdc-9be2-4c59-b8b4-68d5ed29046c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:06.032338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8f682cdc-9be2-4c59-b8b4-68d5ed29046c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:06.033518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8f682cdc-9be2-4c59-b8b4-68d5ed29046c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:06.099243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 606/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:06.123856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c89fa-3ae5-4c3d-b279-4551eecd04ac None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:06.228337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c89fa-3ae5-4c3d-b279-4551eecd04ac None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:06.262979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fed197-ccf9-4afa-a2a3-6df4a0803940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:06.263691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fed197-ccf9-4afa-a2a3-6df4a0803940 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:06.263975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fed197-ccf9-4afa-a2a3-6df4a0803940 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:06.264238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fed197-ccf9-4afa-a2a3-6df4a0803940 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:06.274038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fed197-ccf9-4afa-a2a3-6df4a0803940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-18fed197-ccf9-4afa-a2a3-6df4a0803940 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:07:06.278763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:06.279961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:06.281122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 602/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:07:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:06.286212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f682cdc-9be2-4c59-b8b4-68d5ed29046c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:06.326004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:06.383301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f682cdc-9be2-4c59-b8b4-68d5ed29046c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:06.416282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:06.416875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:06.417151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:06.417420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:06.418209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:06.418600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:06.420062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:06.428471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 607/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:07:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:06.434624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fed197-ccf9-4afa-a2a3-6df4a0803940 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:06.484689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:06.533165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fed197-ccf9-4afa-a2a3-6df4a0803940 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:06.566659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:06.567309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:06.567588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:06.567859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:06.577561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:06.578986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:06.593118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:06.594135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:06.595082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:06.595082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:06.595082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:06.595082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 603/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:06.599966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:06.696705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:06.700459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4c494-fc9c-4b52-bc88-79e747e407c2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:06.733624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:06.735467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:06.735467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:06.735467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:06.735467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:06.735821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:06.737188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:06.749021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 608/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:06.754706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:06.856372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f57ff-4c86-4d82-9ff1-e3c019bfe401 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:06.857812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:06.889845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:06.890458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:06.890736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:06.891025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:06.898165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:06.899694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:07:06.910505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:06.911595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:06.913186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 604/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:06.917541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:07.016052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99875af5-c082-4bfe-9d18-c8c61420c876 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:07.056849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:07.057447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:07.057727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:07.058062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:07.058725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:07.059045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:07.061097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:07.078375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 609/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 12:07:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:07.084328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:07.191080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa8163-7ea9-43d5-a208-6ad97a0d9cb1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:07.223823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64472e54-23a6-4ca4-b088-5819806f7c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:07.224415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64472e54-23a6-4ca4-b088-5819806f7c6f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:07.224697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64472e54-23a6-4ca4-b088-5819806f7c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:07.224963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64472e54-23a6-4ca4-b088-5819806f7c6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:07.255067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:07.292059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64472e54-23a6-4ca4-b088-5819806f7c6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:07.294039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64472e54-23a6-4ca4-b088-5819806f7c6f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:07.392362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 605/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:07:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:07:07.398322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:07.417500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:07.497020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7cc66-641c-47f4-8a5f-fdb6fadecca2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:07.532039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:07.532661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:07.532946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:07.533221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:07.543128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:07.545234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:07.559026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:07.560835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:07.561797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:07.561797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:07.561797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:07.561797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 610/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:07.566792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64472e54-23a6-4ca4-b088-5819806f7c6f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:07.628861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:07.666231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64472e54-23a6-4ca4-b088-5819806f7c6f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:07.698422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:07.699028 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:07.699321 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:07.699594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:07.700995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:07.701295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:07.702642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:07.714080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 606/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:07.719809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:07.787343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:07.817016 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cada0-ef1c-4e9f-b13e-921099e4cac2 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:07.850979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:07.851614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:07.851891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:07.852159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:07.859111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:07.860584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:07:07.872733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:07.877693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:07.878759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 611/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:07:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:07.879746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:07.979129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8fe8-3a86-489a-aacb-27b3981adec9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:08.002960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:08.014071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8437e-6858-47b0-8d0f-9e195464dd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:08.014738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8437e-6858-47b0-8d0f-9e195464dd30 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:08.015041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8437e-6858-47b0-8d0f-9e195464dd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:08.015320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8437e-6858-47b0-8d0f-9e195464dd30 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:08.016009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8437e-6858-47b0-8d0f-9e195464dd30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:08.016315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8437e-6858-47b0-8d0f-9e195464dd30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:08.017742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8437e-6858-47b0-8d0f-9e195464dd30 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:08.028998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 607/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:07:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:08.035149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:08.135257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd709742-6c8e-4694-92dc-25e6140a2f1f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:08.167726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:08.174006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c88a-19ce-4ab0-a433-14b68d7fbb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:08.174600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c88a-19ce-4ab0-a433-14b68d7fbb49 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:08.174877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c88a-19ce-4ab0-a433-14b68d7fbb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:08.175180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c88a-19ce-4ab0-a433-14b68d7fbb49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:08.175956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c88a-19ce-4ab0-a433-14b68d7fbb49 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=c3ee3d70937740b8adcb16ab57dc3031) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:08.176261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c88a-19ce-4ab0-a433-14b68d7fbb49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:08.177680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c88a-19ce-4ab0-a433-14b68d7fbb49 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:08.189063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 612/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:07:08 2022] GET /identity/v3/role_assignments?user.id=c3ee3d70937740b8adcb16ab57dc3031&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:10.181096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399c6cb-d750-4a91-b332-b082723362df None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:10.181657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399c6cb-d750-4a91-b332-b082723362df None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:10.181751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399c6cb-d750-4a91-b332-b082723362df None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:10.183044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 608/1220] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:10.188271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a363-766c-41c6-a970-5f3e4857150a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:10.188696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a363-766c-41c6-a970-5f3e4857150a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:10.188817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a363-766c-41c6-a970-5f3e4857150a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:10.207252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-81b2a363-766c-41c6-a970-5f3e4857150a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:10.208415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-81b2a363-766c-41c6-a970-5f3e4857150a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:10.277089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 613/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:10.299321 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399c6cb-d750-4a91-b332-b082723362df None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:10.402280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399c6cb-d750-4a91-b332-b082723362df None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:10.434386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab31b6-2dc4-4085-a9e5-634ee95981e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:10.434992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab31b6-2dc4-4085-a9e5-634ee95981e2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:10.435296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab31b6-2dc4-4085-a9e5-634ee95981e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:10.435562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab31b6-2dc4-4085-a9e5-634ee95981e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:10.445595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab31b6-2dc4-4085-a9e5-634ee95981e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-52ab31b6-2dc4-4085-a9e5-634ee95981e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:07:10.450706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:10.451935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:10.453168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 609/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:07:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:10.457858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2a363-766c-41c6-a970-5f3e4857150a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:10.555095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2a363-766c-41c6-a970-5f3e4857150a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:10.577615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:10.587787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:10.588379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:10.588654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:10.588923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:10.589579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] RBAC: Adding query filter params (name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:10.589878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:10.591794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:10.599333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 614/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:07:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:10.605122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab31b6-2dc4-4085-a9e5-634ee95981e2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:10.647924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:10.702238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab31b6-2dc4-4085-a9e5-634ee95981e2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:10.734773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:10.735420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:10.735728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:10.736000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:10.746384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:10.747943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:10.761888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:10.763062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:10.764388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:10.764388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:10.764388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:10.764388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 610/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 12:07:10.768851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:10.870488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb4dc9-79e3-4f1a-804d-a2631477242a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:10.903492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:10.904314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:10.904595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:10.904868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:10.905503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:10.905804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:10.907216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:10.919333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 615/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:10.925024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:11.019019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:11.025367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c05ef-003d-44b6-8535-8e0e9ef995a7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:11.058133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:11.058759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:11.059070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:11.059390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:11.066129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:11.067662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:07:11.079635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:11.080785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:11.082060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 611/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:11.086736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:11.135238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:11.186788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a9298-35f0-411b-bdf8-22093b509abf None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:11.221965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:11.222619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:11.222832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:11.223129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:11.223854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:11.224154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:11.225573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:11.243520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 616/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 12:07:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:11.250312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:11.350439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39e4a2-d8a6-4cdc-9e00-cceefb2c2d49 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:11.384091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:11.384703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:11.384977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:11.385247 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:11.396510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:11.397922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:11.412334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:11.413705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:11.414643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:11.414643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:11.414643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:11.414643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 612/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:11.420188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:11.520771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208490-0454-4dc0-9652-51e986ed3040 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:11.523097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:11.554763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:11.555412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:11.555690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:11.555959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:11.556584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:11.556883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:11.558267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:11.569230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 617/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:11.574689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:11.584097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:11.673473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84295483-0764-440a-85cf-04a49d4c39e9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:11.706618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:11.707287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:11.707562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:11.707827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:11.714515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:11.716595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:07:11.727598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:11.730604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:11.732246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 613/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:07:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:11.734820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:11.833553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b75e8-8863-4ddc-82cc-45ac8e7d6d8f None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:11.866445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a81b67-29e7-4956-95c1-c60d800d4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:11.867121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a81b67-29e7-4956-95c1-c60d800d4933 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:11.867403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a81b67-29e7-4956-95c1-c60d800d4933 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:11.867676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a81b67-29e7-4956-95c1-c60d800d4933 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:11.868357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a81b67-29e7-4956-95c1-c60d800d4933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:11.868660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a81b67-29e7-4956-95c1-c60d800d4933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:11.870345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a81b67-29e7-4956-95c1-c60d800d4933 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:11.883120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 618/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:07:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:11.889602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:11.953981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:11.987379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d17744-bf16-4812-b7b2-55ba3c424847 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:12.020633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44473-4fe0-4cd5-b9c9-de83595aae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:12.021310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44473-4fe0-4cd5-b9c9-de83595aae1d None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:12.021584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44473-4fe0-4cd5-b9c9-de83595aae1d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:12.021855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44473-4fe0-4cd5-b9c9-de83595aae1d None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/c3ee3d70937740b8adcb16ab57dc3031/roles/2e4b771225e74dfc974d2ba9b872c77f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:12.035790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b44473-4fe0-4cd5-b9c9-de83595aae1d None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=c3ee3d70937740b8adcb16ab57dc3031, role_id=2e4b771225e74dfc974d2ba9b872c77f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:12.037206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b44473-4fe0-4cd5-b9c9-de83595aae1d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:12.115016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 614/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:07:11 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/c3ee3d70937740b8adcb16ab57dc3031/roles/2e4b771225e74dfc974d2ba9b872c77f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:07:14.108498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f2cee-a343-4232-80fe-aeaaa61c84e4 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:14.108916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f2cee-a343-4232-80fe-aeaaa61c84e4 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:14.108916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f2cee-a343-4232-80fe-aeaaa61c84e4 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:14.110335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 619/1233] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:14.115539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44425f6-12bd-4188-8daf-20c58aa3446e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:14.115960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44425f6-12bd-4188-8daf-20c58aa3446e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:14.116090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44425f6-12bd-4188-8daf-20c58aa3446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:14.134577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c44425f6-12bd-4188-8daf-20c58aa3446e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:14.135798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c44425f6-12bd-4188-8daf-20c58aa3446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:14.203716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 615/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:14.227724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f2cee-a343-4232-80fe-aeaaa61c84e4 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:14.252630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:14.328750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f2cee-a343-4232-80fe-aeaaa61c84e4 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:14.361863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787cf52-9c24-4ffd-9922-fc65074de245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:14.362406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787cf52-9c24-4ffd-9922-fc65074de245 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:14.362679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787cf52-9c24-4ffd-9922-fc65074de245 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:14.362946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787cf52-9c24-4ffd-9922-fc65074de245 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:14.374129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787cf52-9c24-4ffd-9922-fc65074de245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-f787cf52-9c24-4ffd-9922-fc65074de245 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:07:14.378810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:14.380011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:14.381100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 620/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Sun Aug 7 12:07:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:14.385888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44425f6-12bd-4188-8daf-20c58aa3446e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:14.435654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:14.487565 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44425f6-12bd-4188-8daf-20c58aa3446e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:14.521069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:14.521597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:14.521873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:14.522141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:14.522793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:14.523129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:14.524510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:14.532332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 616/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Sun Aug 7 12:07:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:14.538203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787cf52-9c24-4ffd-9922-fc65074de245 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:14.621038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:14.637005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787cf52-9c24-4ffd-9922-fc65074de245 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:14.670377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:14.671025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:14.671319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:14.671585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:14.681438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:14.682908 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:14.697830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:14.698801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:14.700374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:14.700374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:14.700374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:14.700374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 621/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:14.705635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:14.805684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486cace-c16a-4d11-835c-8d11c18e1365 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:14.814646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:14.839477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:14.840219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:14.841110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:14.841380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:14.842004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:14.842301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:14.843671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:14.855404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 617/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:14.861198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:14.963212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752aa4-e456-4e95-ba34-7d5349da1a8e None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:14.997275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:14.997911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:14.998171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:14.998444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:15.001244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:15.006248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:15.007791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:07:15.018775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:15.019813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:15.020941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 622/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:15.025799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:15.126933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2d431-14ab-4790-8566-8e45b581485a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:15.161070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:15.161665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:15.161943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:15.162215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:15.162929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:15.163264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:15.164672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:15.181918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 618/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 12:07:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:15.188080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:15.286940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6f92e-5523-43a0-89e8-954ae707438a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:15.321526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16597bd-c272-4f6a-b538-4d1e681ba337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:15.322120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16597bd-c272-4f6a-b538-4d1e681ba337 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:15.322401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16597bd-c272-4f6a-b538-4d1e681ba337 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:15.322671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16597bd-c272-4f6a-b538-4d1e681ba337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:15.383506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:15.389965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16597bd-c272-4f6a-b538-4d1e681ba337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:15.391469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16597bd-c272-4f6a-b538-4d1e681ba337 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:15.486586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 623/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:07:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:07:15.492551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:15.520144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:15.591750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24ec21-6025-40a2-ad01-3ab22640b0e9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:15.625712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:15.626328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:15.626628 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:15.626899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:15.637577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:15.641593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:15.653522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:15.654640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:15.655619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:15.655619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:15.655619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:15.655619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 619/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:15.660422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16597bd-c272-4f6a-b538-4d1e681ba337 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:15.753115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:15.758747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16597bd-c272-4f6a-b538-4d1e681ba337 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:15.792065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:15.792655 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:15.792939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:15.793215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:15.793882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:15.794184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:15.795620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:15.807622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 624/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:15.813832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:15.894022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:15.915020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d897585-8613-4c0c-bc3c-e84e9ae64e7b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:15.948700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:15.949319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:15.949603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:15.950535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:15.957200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:15.958626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:07:15.969842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:15.975987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:15.977099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 620/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:15.977836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:16.077308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab78fdd-635a-4389-b661-9851fcb18358 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:16.111927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb9bdc-1c8d-4cd5-a89f-54654e219b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:16.112614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9bdc-1c8d-4cd5-a89f-54654e219b66 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:16.112906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9bdc-1c8d-4cd5-a89f-54654e219b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:16.113171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9bdc-1c8d-4cd5-a89f-54654e219b66 None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:16.113865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb9bdc-1c8d-4cd5-a89f-54654e219b66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:16.114168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb9bdc-1c8d-4cd5-a89f-54654e219b66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:16.115662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb9bdc-1c8d-4cd5-a89f-54654e219b66 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:16.128504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 625/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:16.134555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:16.233125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b7a4-60fc-4a37-a79a-8e2c0724872a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:16.266565 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff5d90-8d5d-4b4d-aaec-eb59e6557e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:16.267310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff5d90-8d5d-4b4d-aaec-eb59e6557e80 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:16.267692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff5d90-8d5d-4b4d-aaec-eb59e6557e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:16.267855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff5d90-8d5d-4b4d-aaec-eb59e6557e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:16.268571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff5d90-8d5d-4b4d-aaec-eb59e6557e80 None admin] RBAC: Adding query filter params (role.id=2e4b771225e74dfc974d2ba9b872c77f, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=c3ee3d70937740b8adcb16ab57dc3031) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:16.268867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff5d90-8d5d-4b4d-aaec-eb59e6557e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:16.270257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff5d90-8d5d-4b4d-aaec-eb59e6557e80 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:16.277267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 621/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:07:16 2022] GET /identity/v3/role_assignments?user.id=c3ee3d70937740b8adcb16ab57dc3031&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=2e4b771225e74dfc974d2ba9b872c77f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:18.344577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f2348-d826-4e17-834d-c21490afc62f None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:18.345097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f2348-d826-4e17-834d-c21490afc62f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:18.345209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f2348-d826-4e17-834d-c21490afc62f None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:18.346524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 626/1247] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:18.351718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ee63-0b8f-428b-ab46-5b0932b246e6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:18.352145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ee63-0b8f-428b-ab46-5b0932b246e6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:18.352294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ee63-0b8f-428b-ab46-5b0932b246e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:18.358681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:18.370704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-79f1ee63-0b8f-428b-ab46-5b0932b246e6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:18.372730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-79f1ee63-0b8f-428b-ab46-5b0932b246e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:18.438032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 622/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:18.462069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f2348-d826-4e17-834d-c21490afc62f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:18.480452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:18.565708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f2348-d826-4e17-834d-c21490afc62f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:18.598311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15e263-4dd6-43a6-ae89-63db65451875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:18.598922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15e263-4dd6-43a6-ae89-63db65451875 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:18.599222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15e263-4dd6-43a6-ae89-63db65451875 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:18.599490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15e263-4dd6-43a6-ae89-63db65451875 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:18.609714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15e263-4dd6-43a6-ae89-63db65451875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-5e15e263-4dd6-43a6-ae89-63db65451875 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:07:18.614338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:18.615515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:18.616589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 627/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:18.623149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1ee63-0b8f-428b-ab46-5b0932b246e6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:18.720807 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1ee63-0b8f-428b-ab46-5b0932b246e6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:18.750672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:18.755531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:18.756127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:18.756401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:18.756669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:18.757322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:18.757620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:18.759073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:18.767237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 623/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:18.773310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15e263-4dd6-43a6-ae89-63db65451875 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:18.850187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:18.873070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15e263-4dd6-43a6-ae89-63db65451875 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:18.906926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:18.907541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:18.907844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:18.908097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:18.917823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:18.927700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:18.934227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:18.935435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:18.936581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:18.936581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:18.936581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:18.936581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 628/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:18.941196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:19.040769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f963983-5ca2-43a6-b09a-1ce6cc9f0700 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:19.073654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:19.074242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:19.074518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:19.074819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:19.075503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:19.075802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:19.077148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:19.088483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 624/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:19.094826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:19.193480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ce8f0-2551-417c-bc3a-fdaaa53a49bd None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:19.219518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:19.226944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:19.227582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:19.227858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:19.228126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:19.235282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:19.236766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:07:19.247904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:19.249040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:19.250156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 629/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:19.254957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:19.310384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:19.355799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd123b8-4666-4f62-ad23-7e411381c6cd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:19.389193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:19.389830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:19.390107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:19.390381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:19.391127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:19.391433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:19.392942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:19.409823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 625/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 12:07:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:19.415715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:19.516273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846707f-1278-4eff-bbf5-cde95443dadc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:19.548496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b415fdc-7008-4386-aebb-9543a5e2a371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:19.549089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b415fdc-7008-4386-aebb-9543a5e2a371 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:19.549365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b415fdc-7008-4386-aebb-9543a5e2a371 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:19.549631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b415fdc-7008-4386-aebb-9543a5e2a371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:19.603586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:19.615831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b415fdc-7008-4386-aebb-9543a5e2a371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:19.617295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b415fdc-7008-4386-aebb-9543a5e2a371 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:19.717575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 630/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:07:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:07:19.723464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:19.821792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0a09-9daf-4d94-b2e9-e44a8d3e2b1c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:19.836556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:19.860491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:19.861119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:19.861398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:19.861667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:19.872376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:19.873807 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:19.888509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:19.892519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:19.893656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:19.893656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:19.893656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:19.893656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 626/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:19.895286 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b415fdc-7008-4386-aebb-9543a5e2a371 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:19.980714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:19.992970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b415fdc-7008-4386-aebb-9543a5e2a371 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:20.025882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:20.026475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:20.026754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:20.027056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:20.027688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:20.028183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:20.029752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:20.040573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 631/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:20.047107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:20.146947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbd9ea-686e-4c19-9eb3-fbaf870667ae None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:20.179557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:20.180179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:20.180453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:20.180720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:20.187844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:20.189668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:07:20.200590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:20.201674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:20.207690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 627/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:20.208232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:20.305916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078866a3-8ba0-48cb-9acf-91c4750e3e48 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:20.338880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c953e-4321-4e44-b682-4f38b521d47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:20.339496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c953e-4321-4e44-b682-4f38b521d47c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:20.339771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c953e-4321-4e44-b682-4f38b521d47c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:20.340037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c953e-4321-4e44-b682-4f38b521d47c None admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:20.341244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c953e-4321-4e44-b682-4f38b521d47c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:20.341547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c953e-4321-4e44-b682-4f38b521d47c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:20.342982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c953e-4321-4e44-b682-4f38b521d47c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:20.354539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 632/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:07:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:07:20.361351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:20.388517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:20.462840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce852a-ab6c-42be-8526-0829d4c331ca None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:20.498756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecea11-4553-4f12-8492-7b53dd58ec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:20.499386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea11-4553-4f12-8492-7b53dd58ec9c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:20.499667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea11-4553-4f12-8492-7b53dd58ec9c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:20.499941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea11-4553-4f12-8492-7b53dd58ec9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:20.500682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecea11-4553-4f12-8492-7b53dd58ec9c None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=c3ee3d70937740b8adcb16ab57dc3031) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:20.500989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecea11-4553-4f12-8492-7b53dd58ec9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:20.502432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecea11-4553-4f12-8492-7b53dd58ec9c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:20.507876 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 628/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:07:20 2022] GET /identity/v3/role_assignments?user.id=c3ee3d70937740b8adcb16ab57dc3031&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=f72d3388f13149b483355331380f040b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:22.475123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff6b7-9de0-4e34-9947-ed0a43323f4a None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:22.475682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff6b7-9de0-4e34-9947-ed0a43323f4a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:22.475682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff6b7-9de0-4e34-9947-ed0a43323f4a None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:22.477012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 633/1261] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:22.482060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374adee-66bd-4867-a607-1e427b8c2567 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:22.482451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374adee-66bd-4867-a607-1e427b8c2567 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:22.482619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374adee-66bd-4867-a607-1e427b8c2567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:22.500884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4374adee-66bd-4867-a607-1e427b8c2567 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:22.502090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4374adee-66bd-4867-a607-1e427b8c2567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:22.568672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 629/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:22.591450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceff6b7-9de0-4e34-9947-ed0a43323f4a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:22.602239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:22.693645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceff6b7-9de0-4e34-9947-ed0a43323f4a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:22.726493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7834841-9395-4bb4-b979-85f012664f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:22.727070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7834841-9395-4bb4-b979-85f012664f42 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:22.727352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7834841-9395-4bb4-b979-85f012664f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:22.727622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7834841-9395-4bb4-b979-85f012664f42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:22.737360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7834841-9395-4bb4-b979-85f012664f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-f7834841-9395-4bb4-b979-85f012664f42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:07:22.741984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:22.743130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:22.744271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 634/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:07:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:22.752045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374adee-66bd-4867-a607-1e427b8c2567 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:22.772869 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:22.850661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374adee-66bd-4867-a607-1e427b8c2567 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:22.884030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:22.884625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:22.884903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:22.885172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:22.885826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:22.886124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:22.888136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:22.897123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 630/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:22.903136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7834841-9395-4bb4-b979-85f012664f42 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:22.971635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:23.004321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7834841-9395-4bb4-b979-85f012664f42 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:23.036730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:23.037345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:23.037623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:23.037898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:23.048295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:23.051041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:23.065932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:23.067145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:23.068229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:23.068229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:23.068229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:23.068229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 635/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:23.071573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:23.155138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:23.174430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55a4fd-c67e-47f1-92ab-f5bbcb367c01 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:23.208151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:23.208750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:23.209032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:23.209305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:23.209940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:23.210242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:23.211686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:23.223769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 631/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:23.229399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:23.327907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af4e5-9edf-47ec-9f69-e1edb6370fbc None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:23.347337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:23.361294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:23.361902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:23.362178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:23.362446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:23.369951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:23.371406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:07:23.382242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:23.383498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:23.384742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:23.387155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 636/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:23.389112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:23.489655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fca93-5e91-4b5a-a19d-16f328b8ae60 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:23.523337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:23.523940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:23.524242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:23.524516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:23.525233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:23.525534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:23.526977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:23.537027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:23.546108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 632/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 12:07:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:23.551842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:23.650661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c661b-93df-4c9c-97fa-d6a0cb321ba5 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:23.682214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:23.682815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:23.683116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:23.683413 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:23.694178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:23.695675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:23.709788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:23.710967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:23.711966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:23.711966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:23.711966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:23.711966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 637/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:23.716695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:23.815704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f61c8-b679-4dea-acd3-8e2e04962116 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:23.849381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:23.850047 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:23.850322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:23.850590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:23.851249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:23.851545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:23.852949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:23.864372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 633/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:23.870282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:23.901329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:23.968506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41729f90-b9c4-42b6-b241-296233f25c5c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:24.002061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:24.002673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:24.002951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:24.003248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:24.009641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:24.011113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:07:24.022225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:24.023318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:24.024304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 638/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:07:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:24.029279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:24.087488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:24.129837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34117f3-3dc0-4151-8d22-bf2d8c81c0a7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:24.163973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cdce8-8ff6-4040-8a12-8a0e60ac321b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:24.163973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cdce8-8ff6-4040-8a12-8a0e60ac321b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:24.163973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cdce8-8ff6-4040-8a12-8a0e60ac321b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:24.163973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cdce8-8ff6-4040-8a12-8a0e60ac321b None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:24.164605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cdce8-8ff6-4040-8a12-8a0e60ac321b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:24.164911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cdce8-8ff6-4040-8a12-8a0e60ac321b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:24.166370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cdce8-8ff6-4040-8a12-8a0e60ac321b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:24.178968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 634/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:07:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:07:24.184996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:24.274935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:24.284178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3132c-e193-4030-914f-26c85179a8d9 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:24.321487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f931d-e518-4522-a51e-59c9ec7755c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:24.322187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f931d-e518-4522-a51e-59c9ec7755c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:24.322465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f931d-e518-4522-a51e-59c9ec7755c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:24.322736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f931d-e518-4522-a51e-59c9ec7755c0 None admin] PATH_INFO: `/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/c3ee3d70937740b8adcb16ab57dc3031/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:24.324633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f931d-e518-4522-a51e-59c9ec7755c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa20498b52fb441c8734f1813776cf2e, user_id=c3ee3d70937740b8adcb16ab57dc3031, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:24.326173 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f931d-e518-4522-a51e-59c9ec7755c0 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:24.399726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 639/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Sun Aug 7 12:07:24 2022] PUT /identity/v3/projects/aa20498b52fb441c8734f1813776cf2e/users/c3ee3d70937740b8adcb16ab57dc3031/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:07:26.388113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128510-829e-4625-8234-fd21adfd3a0a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:26.388709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128510-829e-4625-8234-fd21adfd3a0a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:26.388709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128510-829e-4625-8234-fd21adfd3a0a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:26.390018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 635/1274] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:26.395517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8287f-3548-4555-a330-8d146043ac0e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:26.396659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8287f-3548-4555-a330-8d146043ac0e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:26.396659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8287f-3548-4555-a330-8d146043ac0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:26.413898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-09b8287f-3548-4555-a330-8d146043ac0e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:26.415124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-09b8287f-3548-4555-a330-8d146043ac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:26.480546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 640/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:26.505158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db128510-829e-4625-8234-fd21adfd3a0a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:26.605126 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db128510-829e-4625-8234-fd21adfd3a0a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:26.623939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:26.640297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845593ae-1b9b-4629-87fe-42637f30322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:26.640888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845593ae-1b9b-4629-87fe-42637f30322c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:26.641165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845593ae-1b9b-4629-87fe-42637f30322c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:26.641439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845593ae-1b9b-4629-87fe-42637f30322c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:26.651189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845593ae-1b9b-4629-87fe-42637f30322c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-845593ae-1b9b-4629-87fe-42637f30322c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 12:07:26.656446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:26.657677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:26.658810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 636/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Sun Aug 7 12:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:26.664557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8287f-3548-4555-a330-8d146043ac0e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:26.744332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:26.761790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8287f-3548-4555-a330-8d146043ac0e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:26.795718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:26.796307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:26.796582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:26.796850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:26.797611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:26.797913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:26.799381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:26.807105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 641/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Sun Aug 7 12:07:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:26.812859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845593ae-1b9b-4629-87fe-42637f30322c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:26.911959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845593ae-1b9b-4629-87fe-42637f30322c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:26.944393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:26.945008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:26.945281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:26.945550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:26.956419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:26.957836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:26.972384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:26.973670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:26.974892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:26.974892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:26.974892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:26.974892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 637/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:26.979244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:27.078924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba1f53-76bb-4b36-a9b0-55ca64dac38d None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:27.111461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:27.112059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:27.112336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:27.112603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:27.113233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:27.113531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:27.114922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:27.127748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 642/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:27.133391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:27.167977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:27.235026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ff432-2638-4284-bb5d-d5af62a7aa0c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:27.269900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:27.270555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:27.289127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:27.289127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] PATH_INFO: `/v3/users/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:27.289127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:27.289127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 12:07:27.290885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:27.292090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:27.293857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 638/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:27.298581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:27.302377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:27.395978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb7d94-48fd-4ad8-a7c0-23a90ff6dd2a None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:27.431165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:27.431699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:27.431980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:27.432253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:27.432971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:27.433278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:27.434828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:27.452320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 643/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Sun Aug 7 12:07:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:27.458033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:27.547418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:27.556673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7a473-d356-4196-a960-60f42a81f7da None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:27.591294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a0d6d-7621-4961-b1d3-e4ebe9c5e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:27.591893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a0d6d-7621-4961-b1d3-e4ebe9c5e72b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:27.592172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a0d6d-7621-4961-b1d3-e4ebe9c5e72b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:27.592440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a0d6d-7621-4961-b1d3-e4ebe9c5e72b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:27.660622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a0d6d-7621-4961-b1d3-e4ebe9c5e72b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:27.662088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a0d6d-7621-4961-b1d3-e4ebe9c5e72b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:27.749916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:27.759770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 639/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Sun Aug 7 12:07:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 12:07:27.765840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:27.826587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:27.863938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771c7db-cc4a-4d51-b476-d5fe5442cb8c None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:27.895958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:27.896572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:27.896847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:27.897114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:27.907337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:27.908761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 12:07:27.923733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:27.925626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 12:07:27.926771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 12:07:27.926771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 12:07:27.926771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:27.926771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 644/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:27.930522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a0d6d-7621-4961-b1d3-e4ebe9c5e72b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:28.028608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a0d6d-7621-4961-b1d3-e4ebe9c5e72b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:28.062284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:28.062882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:28.063192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:28.063462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:28.064093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:28.064392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:28.065747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:28.076875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 640/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Sun Aug 7 12:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:28.082582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:28.181642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d5e73-fa6b-4b27-9cf8-eaa93afa98ab None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:28.195157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:28.214771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:28.215621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:28.215901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:28.216171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:28.222468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:28.223943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 12:07:28.235249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:28.236333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: ERROR keystone.server.flask.application Aug 07 12:07:28.237321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 645/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Sun Aug 7 12:07:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:28.242167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:28.295572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:28.344357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea44f1-42f0-482f-b8a8-217dd8d5664e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:28.376746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b02475-ae19-49e9-a26d-1b179a412dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:28.377337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02475-ae19-49e9-a26d-1b179a412dca None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:28.377613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02475-ae19-49e9-a26d-1b179a412dca None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:28.377879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02475-ae19-49e9-a26d-1b179a412dca None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:28.378556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b02475-ae19-49e9-a26d-1b179a412dca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:28.378882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b02475-ae19-49e9-a26d-1b179a412dca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:28.380601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b02475-ae19-49e9-a26d-1b179a412dca None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:28.393745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 641/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Sun Aug 7 12:07:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:28.399952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:28.499671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dafef-ae5f-49d0-9ac6-91e6fe5a26fe None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:28.533557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3096b0-4b3c-49c1-b96c-0c658713d4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:28.534156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3096b0-4b3c-49c1-b96c-0c658713d4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:28.534434 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3096b0-4b3c-49c1-b96c-0c658713d4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:28.534706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3096b0-4b3c-49c1-b96c-0c658713d4a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:28.535459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3096b0-4b3c-49c1-b96c-0c658713d4a4 None admin] RBAC: Adding query filter params (role.id=f72d3388f13149b483355331380f040b, scope.project.id=aa20498b52fb441c8734f1813776cf2e, user.id=c3ee3d70937740b8adcb16ab57dc3031) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:28.535792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3096b0-4b3c-49c1-b96c-0c658713d4a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:28.537225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3096b0-4b3c-49c1-b96c-0c658713d4a4 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:28.542975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 646/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Sun Aug 7 12:07:28 2022] GET /identity/v3/role_assignments?user.id=c3ee3d70937740b8adcb16ab57dc3031&scope.project.id=aa20498b52fb441c8734f1813776cf2e&role.id=f72d3388f13149b483355331380f040b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:30.576621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098237dc-1bc8-462a-b19e-42b575aeecab None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:30.577216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098237dc-1bc8-462a-b19e-42b575aeecab None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:30.577216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098237dc-1bc8-462a-b19e-42b575aeecab None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:30.578584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 642/1288] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:30.583637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015965bd-b470-4a34-94f8-a5c9d474f7a9 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:30.588383 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015965bd-b470-4a34-94f8-a5c9d474f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:30.588383 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015965bd-b470-4a34-94f8-a5c9d474f7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:30.602313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-015965bd-b470-4a34-94f8-a5c9d474f7a9 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:30.603517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-015965bd-b470-4a34-94f8-a5c9d474f7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:30.633373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:30.670416 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 647/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:30.692172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098237dc-1bc8-462a-b19e-42b575aeecab None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:30.792874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098237dc-1bc8-462a-b19e-42b575aeecab None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:30.803467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:30.826889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:30.827483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:30.827759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:30.828026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:30.828592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:30.829934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:30.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:30.839784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:30.841090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 643/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:07:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:30.845776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015965bd-b470-4a34-94f8-a5c9d474f7a9 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:30.941999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015965bd-b470-4a34-94f8-a5c9d474f7a9 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:30.975185 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60673353-c42c-43a0-88b3-b03ac63f48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:30.975762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673353-c42c-43a0-88b3-b03ac63f48c3 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:30.976039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673353-c42c-43a0-88b3-b03ac63f48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:30.976311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673353-c42c-43a0-88b3-b03ac63f48c3 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:30.976938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60673353-c42c-43a0-88b3-b03ac63f48c3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:30.977236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60673353-c42c-43a0-88b3-b03ac63f48c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:30.978640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60673353-c42c-43a0-88b3-b03ac63f48c3 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:30.983567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 648/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Sun Aug 7 12:07:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:30.989458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:31.089419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f86a-7ba7-48de-b58f-4bb8a5dbe4d1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:31.125698 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72500f1-eea2-4061-a2cd-2d50a70907dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:31.126320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72500f1-eea2-4061-a2cd-2d50a70907dd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:31.126576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72500f1-eea2-4061-a2cd-2d50a70907dd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:31.126845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72500f1-eea2-4061-a2cd-2d50a70907dd None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:31.127495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72500f1-eea2-4061-a2cd-2d50a70907dd None admin] RBAC: Adding query filter params (type=placement) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:31.127797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72500f1-eea2-4061-a2cd-2d50a70907dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:31.129147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72500f1-eea2-4061-a2cd-2d50a70907dd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:31.134439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 644/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Sun Aug 7 12:07:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:33.136300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e364d5-5f7b-49a5-8366-ef0199b1309c None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:33.136809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e364d5-5f7b-49a5-8366-ef0199b1309c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:33.136902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e364d5-5f7b-49a5-8366-ef0199b1309c None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:33.138530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 649/1293] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:33.143547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25224fff-03ce-461f-b2c2-f82c62a055a7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:33.143926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25224fff-03ce-461f-b2c2-f82c62a055a7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:33.144095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25224fff-03ce-461f-b2c2-f82c62a055a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:33.162553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-25224fff-03ce-461f-b2c2-f82c62a055a7 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:33.163786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-25224fff-03ce-461f-b2c2-f82c62a055a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:33.194052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:33.233286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 645/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:33.255899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e364d5-5f7b-49a5-8366-ef0199b1309c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:33.286955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:33.357791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e364d5-5f7b-49a5-8366-ef0199b1309c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:33.390386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f134ec-b629-4326-ab1a-56032270fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:33.390973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f134ec-b629-4326-ab1a-56032270fe55 None admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:33.391281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f134ec-b629-4326-ab1a-56032270fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:33.391548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f134ec-b629-4326-ab1a-56032270fe55 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:33.392185 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f134ec-b629-4326-ab1a-56032270fe55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:33.393538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f134ec-b629-4326-ab1a-56032270fe55 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:33.479606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 650/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Sun Aug 7 12:07:33 2022] POST /identity/v3/services => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:07:35.472850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6505b7-d3d7-441a-ad8f-8009533f9213 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:35.473384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6505b7-d3d7-441a-ad8f-8009533f9213 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:35.473384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6505b7-d3d7-441a-ad8f-8009533f9213 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:35.474739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 646/1296] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:35.479878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573af9-04fa-4036-b669-7aaf1d05b03b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:35.480270 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573af9-04fa-4036-b669-7aaf1d05b03b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:35.480443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35573af9-04fa-4036-b669-7aaf1d05b03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:35.499170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-35573af9-04fa-4036-b669-7aaf1d05b03b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:35.500349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-35573af9-04fa-4036-b669-7aaf1d05b03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:35.570868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 651/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:35.594086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6505b7-d3d7-441a-ad8f-8009533f9213 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:35.693289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6505b7-d3d7-441a-ad8f-8009533f9213 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:35.727426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:35.727980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:35.728257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:35.728529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] PATH_INFO: `/v3/services/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:35.729106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:35.730446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:35.739040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:35.746391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:35.747478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 647/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:07:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:35.748464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35573af9-04fa-4036-b669-7aaf1d05b03b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:35.765295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:35.844572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35573af9-04fa-4036-b669-7aaf1d05b03b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:35.877480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:35.878089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:35.878399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:35.878673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:35.879350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:35.879657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:35.881035 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:35.886073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 652/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Sun Aug 7 12:07:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:35.892842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:35.924122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:35.991428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930485b-b4e8-499b-99d8-5fc70576fbce None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:36.025113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b853b9-9661-4430-979f-df6261b5382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:36.025705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b853b9-9661-4430-979f-df6261b5382a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:36.025980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b853b9-9661-4430-979f-df6261b5382a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:36.026251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b853b9-9661-4430-979f-df6261b5382a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:36.026966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b853b9-9661-4430-979f-df6261b5382a None admin] RBAC: Adding query filter params (interface=public, service_id=d3ba3b3b8cd74224894c14f2c3ac1d9e, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:36.027341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b853b9-9661-4430-979f-df6261b5382a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:36.029522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b853b9-9661-4430-979f-df6261b5382a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:36.034384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 648/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Sun Aug 7 12:07:35 2022] GET /identity/v3/endpoints?service_id=d3ba3b3b8cd74224894c14f2c3ac1d9e&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:36.040047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:36.125133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:36.143275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af6b4-5bf1-4f79-b88c-256c1f7113e8 None admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:36.175387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e8daf-195a-4d62-bb03-ab4a932e8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:36.175983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e8daf-195a-4d62-bb03-ab4a932e8f49 None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:36.176262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e8daf-195a-4d62-bb03-ab4a932e8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:36.176552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e8daf-195a-4d62-bb03-ab4a932e8f49 None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:36.177161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e8daf-195a-4d62-bb03-ab4a932e8f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:36.178567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e8daf-195a-4d62-bb03-ab4a932e8f49 None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:36.185887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 653/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Sun Aug 7 12:07:36 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:07:38.180787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbc174-4069-47a9-8b8f-b3466ae46857 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:38.181316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbc174-4069-47a9-8b8f-b3466ae46857 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:38.181316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbc174-4069-47a9-8b8f-b3466ae46857 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:38.182778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 649/1302] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:07:38.188205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040b9e-a920-4d1c-98a8-2a95ea12a317 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:38.188617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040b9e-a920-4d1c-98a8-2a95ea12a317 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:38.188749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040b9e-a920-4d1c-98a8-2a95ea12a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:38.206606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-db040b9e-a920-4d1c-98a8-2a95ea12a317 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:07:38.211071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-db040b9e-a920-4d1c-98a8-2a95ea12a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:07:38.278726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 654/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:07:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 12:07:38.303295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbc174-4069-47a9-8b8f-b3466ae46857 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:38.401507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:38.408957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbc174-4069-47a9-8b8f-b3466ae46857 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:38.442271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:38.442887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:38.443196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:38.443467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:38.444061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:38.445411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 12:07:38.454292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 12:07:38.455557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Aug 07 12:07:38.456720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 650/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Sun Aug 7 12:07:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 12:07:38.461334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040b9e-a920-4d1c-98a8-2a95ea12a317 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:38.537858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:38.559333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040b9e-a920-4d1c-98a8-2a95ea12a317 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:38.592633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b5a0f-a7a2-48f7-a567-daf8be8fe06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:38.593225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b5a0f-a7a2-48f7-a567-daf8be8fe06d None admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:38.593507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b5a0f-a7a2-48f7-a567-daf8be8fe06d None admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:38.593783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b5a0f-a7a2-48f7-a567-daf8be8fe06d None admin] PATH_INFO: `/v3/services` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:38.594404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b5a0f-a7a2-48f7-a567-daf8be8fe06d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:07:38.594729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b5a0f-a7a2-48f7-a567-daf8be8fe06d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:38.596130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b5a0f-a7a2-48f7-a567-daf8be8fe06d None admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:38.601273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 655/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Sun Aug 7 12:07:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:07:38.606883 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:07:38.706214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fff96-4629-4369-b261-7fc3c9287a37 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:07:38.739018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e4a16-e88d-4c5e-bbdd-080231a67158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:07:38.739622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4a16-e88d-4c5e-bbdd-080231a67158 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:07:38.739903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4a16-e88d-4c5e-bbdd-080231a67158 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:07:38.740217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4a16-e88d-4c5e-bbdd-080231a67158 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:07:38.740865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4a16-e88d-4c5e-bbdd-080231a67158 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:07:38.742216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4a16-e88d-4c5e-bbdd-080231a67158 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:07:38.820344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 651/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:07:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:07:48.631181 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=30 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:07:48.815388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:37.094904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0366-c4ff-4384-84ba-e482037524c5 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:37.095877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0366-c4ff-4384-84ba-e482037524c5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:37.096362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0366-c4ff-4384-84ba-e482037524c5 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:37.103244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fcc1e-0e25-46d2-b2ae-2f6645cfa295 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:37.106519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fcc1e-0e25-46d2-b2ae-2f6645cfa295 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:37.106519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fcc1e-0e25-46d2-b2ae-2f6645cfa295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:37.107192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 656/1307] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:13:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:37.246404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-002fcc1e-0e25-46d2-b2ae-2f6645cfa295 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:37.250758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-002fcc1e-0e25-46d2-b2ae-2f6645cfa295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:37.423643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 652/1308] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:13:37.474193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d9826-787e-4fe1-851e-d5b17d7b3f27 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:37.474723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d9826-787e-4fe1-851e-d5b17d7b3f27 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:37.475215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d9826-787e-4fe1-851e-d5b17d7b3f27 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:37.479714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 657/1309] 10.222.0.30 () {54 vars in 965 bytes} [Sun Aug 7 12:13:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:37.487122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea81ea0-3062-4902-8a1e-5fbffc98de7c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:37.487644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea81ea0-3062-4902-8a1e-5fbffc98de7c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:37.488062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea81ea0-3062-4902-8a1e-5fbffc98de7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:37.580007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-eea81ea0-3062-4902-8a1e-5fbffc98de7c None None] MFA Rules not processed for user `f7aa31ea778140189b041939e3f01aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:37.581493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eea81ea0-3062-4902-8a1e-5fbffc98de7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:37.605401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:37.671296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 653/1310] 10.222.0.30 () {58 vars in 1062 bytes} [Sun Aug 7 12:13:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:13:37.677562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d9826-787e-4fe1-851e-d5b17d7b3f27 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:13:37.679579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a46d9826-787e-4fe1-851e-d5b17d7b3f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:37.790817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d9826-787e-4fe1-851e-d5b17d7b3f27 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:13:37.866010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100344ca-6c6a-45ce-8a42-ca87c129caf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:13:37.866542 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100344ca-6c6a-45ce-8a42-ca87c129caf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:37.866817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100344ca-6c6a-45ce-8a42-ca87c129caf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:37.867123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100344ca-6c6a-45ce-8a42-ca87c129caf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:37.868811 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-100344ca-6c6a-45ce-8a42-ca87c129caf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:37.930543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:37.991997 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100344ca-6c6a-45ce-8a42-ca87c129caf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:13:37.993562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100344ca-6c6a-45ce-8a42-ca87c129caf1 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:13:38.015404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 658/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:13:40.460259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c2bc0-f56b-4f8f-a234-c2b70461df2a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:40.460738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c2bc0-f56b-4f8f-a234-c2b70461df2a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:40.460819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c2bc0-f56b-4f8f-a234-c2b70461df2a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:40.462325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 654/1312] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:40.468008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020727a4-6c43-4c60-95e4-c05f2e7e392c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:40.468427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020727a4-6c43-4c60-95e4-c05f2e7e392c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:40.468590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020727a4-6c43-4c60-95e4-c05f2e7e392c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:40.543134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-020727a4-6c43-4c60-95e4-c05f2e7e392c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:40.544360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-020727a4-6c43-4c60-95e4-c05f2e7e392c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:40.651161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 659/1313] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:13:40.685601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c2bc0-f56b-4f8f-a234-c2b70461df2a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:13:40.687242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-985c2bc0-f56b-4f8f-a234-c2b70461df2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:40.801130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c2bc0-f56b-4f8f-a234-c2b70461df2a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:13:40.875909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:40.878515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc8f6d-f655-4adf-b034-7a458dd87fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:13:40.879155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc8f6d-f655-4adf-b034-7a458dd87fbf service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:40.879435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc8f6d-f655-4adf-b034-7a458dd87fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:40.879706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc8f6d-f655-4adf-b034-7a458dd87fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:40.881295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cadc8f6d-f655-4adf-b034-7a458dd87fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:40.991106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc8f6d-f655-4adf-b034-7a458dd87fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:13:40.992447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc8f6d-f655-4adf-b034-7a458dd87fbf service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:13:41.010986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 655/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:13:43.573853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eac41-95f0-40d0-898a-e0e5c2054bde None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:43.574396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eac41-95f0-40d0-898a-e0e5c2054bde None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:43.574396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eac41-95f0-40d0-898a-e0e5c2054bde None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:43.575803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 660/1315] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:43.580803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f61a-6bc0-4621-8f55-750467485393 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:43.581236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f61a-6bc0-4621-8f55-750467485393 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:43.581375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f61a-6bc0-4621-8f55-750467485393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:43.648067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b354f61a-6bc0-4621-8f55-750467485393 None None] MFA Rules not processed for user `18a66efba8ca4c218281db23e0303fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:43.649301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b354f61a-6bc0-4621-8f55-750467485393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:43.702633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:43.747988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 656/1316] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:13:43.788165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eac41-95f0-40d0-898a-e0e5c2054bde None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:13:43.789716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-384eac41-95f0-40d0-898a-e0e5c2054bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:43.800096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:43.891884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eac41-95f0-40d0-898a-e0e5c2054bde None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:13:43.962225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edde2a-172c-4544-bc42-2917b723b4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:13:43.963611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edde2a-172c-4544-bc42-2917b723b4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:43.963898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edde2a-172c-4544-bc42-2917b723b4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:43.964177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edde2a-172c-4544-bc42-2917b723b4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:43.965817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-07edde2a-172c-4544-bc42-2917b723b4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:44.026977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:44.072447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edde2a-172c-4544-bc42-2917b723b4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:13:44.073894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edde2a-172c-4544-bc42-2917b723b4f8 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:13:44.094005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 661/1317] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:13:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:13:46.949004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ede9e-3b5b-4ec5-a8dc-412f12b0f17f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:46.949509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ede9e-3b5b-4ec5-a8dc-412f12b0f17f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:46.949509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ede9e-3b5b-4ec5-a8dc-412f12b0f17f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:46.950813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 657/1318] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:46.956120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaf64a-1efd-4bf1-9be0-81cd253ef2d8 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:46.956498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaf64a-1efd-4bf1-9be0-81cd253ef2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:46.956671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaf64a-1efd-4bf1-9be0-81cd253ef2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:47.021203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3caaf64a-1efd-4bf1-9be0-81cd253ef2d8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:47.022394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3caaf64a-1efd-4bf1-9be0-81cd253ef2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:47.115886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 662/1319] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:13:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:13:47.155203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ede9e-3b5b-4ec5-a8dc-412f12b0f17f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:13:47.156719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2d4ede9e-3b5b-4ec5-a8dc-412f12b0f17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:47.262001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ede9e-3b5b-4ec5-a8dc-412f12b0f17f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:13:47.333989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a4676-ac6e-4746-aed1-ed49f62f8f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:13:47.334586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a4676-ac6e-4746-aed1-ed49f62f8f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:47.334866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a4676-ac6e-4746-aed1-ed49f62f8f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:47.335167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a4676-ac6e-4746-aed1-ed49f62f8f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:47.336746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6f7a4676-ac6e-4746-aed1-ed49f62f8f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:47.344295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:47.444104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a4676-ac6e-4746-aed1-ed49f62f8f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:13:47.445528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a4676-ac6e-4746-aed1-ed49f62f8f93 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:13:47.463810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 658/1320] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:13:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:13:50.354007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ce55d-528d-4567-8173-4fefd5275176 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:50.354640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ce55d-528d-4567-8173-4fefd5275176 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:50.354640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ce55d-528d-4567-8173-4fefd5275176 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:50.355951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 663/1321] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:50.361134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a6f29-554a-4301-98cf-619878d34431 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:50.361558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a6f29-554a-4301-98cf-619878d34431 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:50.361742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a6f29-554a-4301-98cf-619878d34431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:50.429232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ca1a6f29-554a-4301-98cf-619878d34431 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:50.430427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ca1a6f29-554a-4301-98cf-619878d34431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:50.489800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:50.528607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 659/1322] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:13:50.565044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ce55d-528d-4567-8173-4fefd5275176 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:13:50.566648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2f7ce55d-528d-4567-8173-4fefd5275176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:50.586537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:50.676729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ce55d-528d-4567-8173-4fefd5275176 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:13:50.758281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59336cae-ee5e-4d3a-96a7-507c749a1083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:13:50.758919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336cae-ee5e-4d3a-96a7-507c749a1083 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:50.759878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336cae-ee5e-4d3a-96a7-507c749a1083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:50.760287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336cae-ee5e-4d3a-96a7-507c749a1083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:50.761953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-59336cae-ee5e-4d3a-96a7-507c749a1083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:50.839620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:50.878114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59336cae-ee5e-4d3a-96a7-507c749a1083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:13:50.879587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59336cae-ee5e-4d3a-96a7-507c749a1083 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:13:50.898179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 664/1323] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:13:52.970766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abdd00-60ad-4d6c-9021-093e6554fbe5 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:52.971326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abdd00-60ad-4d6c-9021-093e6554fbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:52.971326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abdd00-60ad-4d6c-9021-093e6554fbe5 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:52.972648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 660/1324] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:52.978445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88866ef5-fbb8-4f3f-ab1b-6b3de769a078 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:52.978980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88866ef5-fbb8-4f3f-ab1b-6b3de769a078 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:52.978980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88866ef5-fbb8-4f3f-ab1b-6b3de769a078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:53.045886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-88866ef5-fbb8-4f3f-ab1b-6b3de769a078 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:53.047132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-88866ef5-fbb8-4f3f-ab1b-6b3de769a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:53.152205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 665/1325] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:13:53.203660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1525-be52-4250-b634-ccb232b3f1f8 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:53.203973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1525-be52-4250-b634-ccb232b3f1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:53.204251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1525-be52-4250-b634-ccb232b3f1f8 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:53.205598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 661/1326] 10.222.0.30 () {54 vars in 965 bytes} [Sun Aug 7 12:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:53.211425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec5806-b932-4edb-8b9e-1aaf70adca86 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:53.211713 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec5806-b932-4edb-8b9e-1aaf70adca86 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:53.212011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec5806-b932-4edb-8b9e-1aaf70adca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:53.244661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:53.289241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d1ec5806-b932-4edb-8b9e-1aaf70adca86 None None] MFA Rules not processed for user `f7aa31ea778140189b041939e3f01aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:53.290467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d1ec5806-b932-4edb-8b9e-1aaf70adca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:53.366066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 666/1327] 10.222.0.30 () {58 vars in 1062 bytes} [Sun Aug 7 12:13:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:13:53.375687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c1525-be52-4250-b634-ccb232b3f1f8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:13:53.377209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6f4c1525-be52-4250-b634-ccb232b3f1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:53.480939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c1525-be52-4250-b634-ccb232b3f1f8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:13:53.558811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f02b32-4aee-4b32-8d3b-a78bbad7a18b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:13:53.559354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f02b32-4aee-4b32-8d3b-a78bbad7a18b service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:53.559629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f02b32-4aee-4b32-8d3b-a78bbad7a18b service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:53.559896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f02b32-4aee-4b32-8d3b-a78bbad7a18b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:53.561466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-08f02b32-4aee-4b32-8d3b-a78bbad7a18b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:53.579083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:53.672573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f02b32-4aee-4b32-8d3b-a78bbad7a18b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:13:53.673994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f02b32-4aee-4b32-8d3b-a78bbad7a18b service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:13:53.692117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 662/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:13:55.983243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15133003-cb71-4ea7-b410-df158b7235dd None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:55.983772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15133003-cb71-4ea7-b410-df158b7235dd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:55.983861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15133003-cb71-4ea7-b410-df158b7235dd None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:55.985161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 667/1329] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:55.990403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c218db7-22ee-4ba5-8f2c-9f7ac3c56323 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:55.990825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c218db7-22ee-4ba5-8f2c-9f7ac3c56323 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:55.990963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c218db7-22ee-4ba5-8f2c-9f7ac3c56323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:56.063590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3c218db7-22ee-4ba5-8f2c-9f7ac3c56323 None None] MFA Rules not processed for user `18a66efba8ca4c218281db23e0303fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:56.065025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3c218db7-22ee-4ba5-8f2c-9f7ac3c56323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:56.136042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:56.168598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 663/1330] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:13:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:13:56.205486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15133003-cb71-4ea7-b410-df158b7235dd None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:13:56.207048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-15133003-cb71-4ea7-b410-df158b7235dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:56.308201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15133003-cb71-4ea7-b410-df158b7235dd None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:13:56.344189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:56.401026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca38b4f-c186-46be-bd96-a6eeb6bd79fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:13:56.401722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38b4f-c186-46be-bd96-a6eeb6bd79fb service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:56.402056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38b4f-c186-46be-bd96-a6eeb6bd79fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:56.402335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38b4f-c186-46be-bd96-a6eeb6bd79fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:56.404028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7ca38b4f-c186-46be-bd96-a6eeb6bd79fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:56.516095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca38b4f-c186-46be-bd96-a6eeb6bd79fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:13:56.517529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca38b4f-c186-46be-bd96-a6eeb6bd79fb service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:13:56.536654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 668/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:13:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:13:59.129455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6d944-e10a-4ec4-9973-e5700f82d3ac None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:59.129916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6d944-e10a-4ec4-9973-e5700f82d3ac None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:59.130020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6d944-e10a-4ec4-9973-e5700f82d3ac None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:59.131357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 664/1332] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:13:59.136878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193e6c5-681b-442d-a608-6b129987fb50 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:59.137330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193e6c5-681b-442d-a608-6b129987fb50 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:59.137435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193e6c5-681b-442d-a608-6b129987fb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:59.199707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:59.202293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-5193e6c5-681b-442d-a608-6b129987fb50 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:13:59.203533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5193e6c5-681b-442d-a608-6b129987fb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:59.299806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 669/1333] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:13:59.336886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6d944-e10a-4ec4-9973-e5700f82d3ac None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:13:59.338493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f7a6d944-e10a-4ec4-9973-e5700f82d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:59.440536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6d944-e10a-4ec4-9973-e5700f82d3ac None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:13:59.516921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663bf5-e198-4785-ab09-1e392ff8c2d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:13:59.517523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663bf5-e198-4785-ab09-1e392ff8c2d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:13:59.517801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663bf5-e198-4785-ab09-1e392ff8c2d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:13:59.518073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663bf5-e198-4785-ab09-1e392ff8c2d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:13:59.519688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-68663bf5-e198-4785-ab09-1e392ff8c2d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:13:59.546187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:13:59.627844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663bf5-e198-4785-ab09-1e392ff8c2d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:13:59.629272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663bf5-e198-4785-ab09-1e392ff8c2d2 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:13:59.651249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 665/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:01.683760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda03075-cfe3-487e-b5ad-b3e9fa338e09 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:01.684299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda03075-cfe3-487e-b5ad-b3e9fa338e09 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:01.684299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda03075-cfe3-487e-b5ad-b3e9fa338e09 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:01.685652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 670/1335] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:01.690677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8e20a-6ab8-41ca-bf58-4459f32a0487 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:01.691161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8e20a-6ab8-41ca-bf58-4459f32a0487 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:01.691258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8e20a-6ab8-41ca-bf58-4459f32a0487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:01.759883 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-52c8e20a-6ab8-41ca-bf58-4459f32a0487 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:01.761104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-52c8e20a-6ab8-41ca-bf58-4459f32a0487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:01.834317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:01.858249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 666/1336] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:01.898871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda03075-cfe3-487e-b5ad-b3e9fa338e09 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:01.900743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-eda03075-cfe3-487e-b5ad-b3e9fa338e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:02.003631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda03075-cfe3-487e-b5ad-b3e9fa338e09 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:02.030901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:02.076066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc55002-81c1-4541-a442-bac080831e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:02.076692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55002-81c1-4541-a442-bac080831e93 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:02.076970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55002-81c1-4541-a442-bac080831e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:02.077270 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55002-81c1-4541-a442-bac080831e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:02.078848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0cc55002-81c1-4541-a442-bac080831e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:02.186099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55002-81c1-4541-a442-bac080831e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:02.187635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55002-81c1-4541-a442-bac080831e93 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:02.205306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 671/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:04.898605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5945c-3351-4164-abea-0b0faa336441 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:04.899272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5945c-3351-4164-abea-0b0faa336441 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:04.899272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5945c-3351-4164-abea-0b0faa336441 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:04.901143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 667/1338] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:04.906110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33616bd-b39a-4fcc-9a11-610a52a8d8d0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:04.906515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33616bd-b39a-4fcc-9a11-610a52a8d8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:04.906652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33616bd-b39a-4fcc-9a11-610a52a8d8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:04.971865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c33616bd-b39a-4fcc-9a11-610a52a8d8d0 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:04.972990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c33616bd-b39a-4fcc-9a11-610a52a8d8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:04.981037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:05.066829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 672/1339] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:05.103474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5945c-3351-4164-abea-0b0faa336441 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:05.105113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1ba5945c-3351-4164-abea-0b0faa336441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:05.206901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5945c-3351-4164-abea-0b0faa336441 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:05.280081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d0fd8-9cf0-438b-9728-975c7a01ef21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:05.280603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d0fd8-9cf0-438b-9728-975c7a01ef21 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:05.280878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d0fd8-9cf0-438b-9728-975c7a01ef21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:05.281148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d0fd8-9cf0-438b-9728-975c7a01ef21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:05.282723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-865d0fd8-9cf0-438b-9728-975c7a01ef21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:05.317413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:05.389525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d0fd8-9cf0-438b-9728-975c7a01ef21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:05.390938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d0fd8-9cf0-438b-9728-975c7a01ef21 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:05.409347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 668/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:09.606755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe495d0-1185-408f-a001-7aec8e67a4d7 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:09.607323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe495d0-1185-408f-a001-7aec8e67a4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:09.607323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe495d0-1185-408f-a001-7aec8e67a4d7 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:09.608668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 673/1341] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:09.613969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26757-6c17-4c27-903a-d9a379672ee0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:09.614421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26757-6c17-4c27-903a-d9a379672ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:09.614565 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26757-6c17-4c27-903a-d9a379672ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:09.681420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6de26757-6c17-4c27-903a-d9a379672ee0 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:09.682649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6de26757-6c17-4c27-903a-d9a379672ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:09.770562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:09.782438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 669/1342] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:09.821798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe495d0-1185-408f-a001-7aec8e67a4d7 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:09.823476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bbe495d0-1185-408f-a001-7aec8e67a4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:09.927214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe495d0-1185-408f-a001-7aec8e67a4d7 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:09.965393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:10.001625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cffb57-b357-4654-bb26-6f621b13cc02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:10.002231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cffb57-b357-4654-bb26-6f621b13cc02 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:10.002514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cffb57-b357-4654-bb26-6f621b13cc02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:10.002787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cffb57-b357-4654-bb26-6f621b13cc02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:10.004411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e8cffb57-b357-4654-bb26-6f621b13cc02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:10.114309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cffb57-b357-4654-bb26-6f621b13cc02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:10.115805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cffb57-b357-4654-bb26-6f621b13cc02 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:10.135012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 674/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:13.191937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec87e9-9ba8-41e8-81e4-8aede877245d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:13.192440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec87e9-9ba8-41e8-81e4-8aede877245d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:13.192440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec87e9-9ba8-41e8-81e4-8aede877245d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:13.193927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 670/1344] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:13.199667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c20f31-ed30-4b07-8575-4be2d94c9e11 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:13.200114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c20f31-ed30-4b07-8575-4be2d94c9e11 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:13.200226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c20f31-ed30-4b07-8575-4be2d94c9e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:13.267316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-71c20f31-ed30-4b07-8575-4be2d94c9e11 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:13.268548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-71c20f31-ed30-4b07-8575-4be2d94c9e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:13.286855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:13.369003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 675/1345] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:14:13.406052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec87e9-9ba8-41e8-81e4-8aede877245d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:13.407731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4bec87e9-9ba8-41e8-81e4-8aede877245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:13.514441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec87e9-9ba8-41e8-81e4-8aede877245d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:13.593939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8bda4-04ce-4e3a-bf7f-a843ae1882c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:13.594555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8bda4-04ce-4e3a-bf7f-a843ae1882c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:13.594832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8bda4-04ce-4e3a-bf7f-a843ae1882c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:13.595137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8bda4-04ce-4e3a-bf7f-a843ae1882c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:13.596865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-63b8bda4-04ce-4e3a-bf7f-a843ae1882c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:13.643386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:13.711964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8bda4-04ce-4e3a-bf7f-a843ae1882c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:13.713488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8bda4-04ce-4e3a-bf7f-a843ae1882c2 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:13.732877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 671/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:18.027751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae53d-cadd-4f81-8dc8-5d455c16ce08 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:18.028292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae53d-cadd-4f81-8dc8-5d455c16ce08 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:18.028292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae53d-cadd-4f81-8dc8-5d455c16ce08 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:18.029648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 676/1347] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:18.035184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea8ed-6c92-46c6-aaf6-83e76f46638a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:18.035656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea8ed-6c92-46c6-aaf6-83e76f46638a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:18.035765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea8ed-6c92-46c6-aaf6-83e76f46638a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:18.108040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fceea8ed-6c92-46c6-aaf6-83e76f46638a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:18.109259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fceea8ed-6c92-46c6-aaf6-83e76f46638a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:18.207193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:18.209920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 672/1348] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:18.250373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefae53d-cadd-4f81-8dc8-5d455c16ce08 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:18.251973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-aefae53d-cadd-4f81-8dc8-5d455c16ce08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:18.359663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefae53d-cadd-4f81-8dc8-5d455c16ce08 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:18.410395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:18.437790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18adf2-6900-45ab-9947-10c89c024631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:18.438404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18adf2-6900-45ab-9947-10c89c024631 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:18.438886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18adf2-6900-45ab-9947-10c89c024631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:18.439371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18adf2-6900-45ab-9947-10c89c024631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:18.441255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8f18adf2-6900-45ab-9947-10c89c024631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:18.550644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18adf2-6900-45ab-9947-10c89c024631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:18.552122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18adf2-6900-45ab-9947-10c89c024631 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:18.570361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 677/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:23.515500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4175-d524-41d9-8f27-d3205f254ddb None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:23.515954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4175-d524-41d9-8f27-d3205f254ddb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:23.516038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4175-d524-41d9-8f27-d3205f254ddb None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:23.517410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 673/1350] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:23.524495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee14269-e1cf-4eb5-b316-101d4916e0f9 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:23.525923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee14269-e1cf-4eb5-b316-101d4916e0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:23.526734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee14269-e1cf-4eb5-b316-101d4916e0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:23.667820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-cee14269-e1cf-4eb5-b316-101d4916e0f9 None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:23.669073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cee14269-e1cf-4eb5-b316-101d4916e0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:23.692240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:23.795434 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 678/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:14:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:27.856698 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9297-02a7-48e6-b0a8-dd59c01c7c34 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:27.857556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9297-02a7-48e6-b0a8-dd59c01c7c34 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:27.857826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9297-02a7-48e6-b0a8-dd59c01c7c34 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:27.859350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 674/1352] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:14:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:27.864532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdd96b-2ec3-40ae-bdbd-e42f4fe33782 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:27.864938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdd96b-2ec3-40ae-bdbd-e42f4fe33782 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:27.865117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdd96b-2ec3-40ae-bdbd-e42f4fe33782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:27.950236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4abdd96b-2ec3-40ae-bdbd-e42f4fe33782 None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:27.951838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4abdd96b-2ec3-40ae-bdbd-e42f4fe33782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:28.033312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:28.073711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 679/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:29.502273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434e1f-f9a5-4124-b34d-19bb32f7c609 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:29.502783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434e1f-f9a5-4124-b34d-19bb32f7c609 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:29.503082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434e1f-f9a5-4124-b34d-19bb32f7c609 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:29.504480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 675/1354] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:29.510315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4b67b-b065-413c-a381-3a355b7ebffa None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:29.511145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4b67b-b065-413c-a381-3a355b7ebffa None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:29.511579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4b67b-b065-413c-a381-3a355b7ebffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:29.588175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8dc4b67b-b065-413c-a381-3a355b7ebffa None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:29.589389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8dc4b67b-b065-413c-a381-3a355b7ebffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:29.696854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 680/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:14:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:14:33.129438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1a48-d80e-4bcc-bb7d-33db2394bb84 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:33.130300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1a48-d80e-4bcc-bb7d-33db2394bb84 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:33.130584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1a48-d80e-4bcc-bb7d-33db2394bb84 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:33.132283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 676/1356] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:33.138666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2a86-39ca-42d2-8bda-74c1a19c45fc None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:33.139137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2a86-39ca-42d2-8bda-74c1a19c45fc None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:33.139234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2a86-39ca-42d2-8bda-74c1a19c45fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:33.189479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:33.227641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e76d2a86-39ca-42d2-8bda-74c1a19c45fc None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:33.229305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e76d2a86-39ca-42d2-8bda-74c1a19c45fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:33.335510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 681/1357] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:33.390893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1548b59-d31b-4dd8-80ae-cd3a34e59374 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:33.391223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1548b59-d31b-4dd8-80ae-cd3a34e59374 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:33.391505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1548b59-d31b-4dd8-80ae-cd3a34e59374 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:33.392917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 677/1358] 10.222.0.30 () {54 vars in 962 bytes} [Sun Aug 7 12:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:33.401937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c4d1-0818-4251-a896-f711297cfdc1 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:33.402226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c4d1-0818-4251-a896-f711297cfdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:33.402498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c4d1-0818-4251-a896-f711297cfdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:33.488253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-16e8c4d1-0818-4251-a896-f711297cfdc1 None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:33.489450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-16e8c4d1-0818-4251-a896-f711297cfdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:33.540962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:33.610646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 682/1359] 10.222.0.30 () {58 vars in 1059 bytes} [Sun Aug 7 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 07 12:14:33.618958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1548b59-d31b-4dd8-80ae-cd3a34e59374 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:33.620708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e1548b59-d31b-4dd8-80ae-cd3a34e59374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:33.771165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1548b59-d31b-4dd8-80ae-cd3a34e59374 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:33.847581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:33.889857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3686119-e04c-4176-b389-45fcb013ba65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:33.890469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686119-e04c-4176-b389-45fcb013ba65 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:33.890743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686119-e04c-4176-b389-45fcb013ba65 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:33.891056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686119-e04c-4176-b389-45fcb013ba65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:33.892610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3686119-e04c-4176-b389-45fcb013ba65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:33.999118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3686119-e04c-4176-b389-45fcb013ba65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:34.000743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3686119-e04c-4176-b389-45fcb013ba65 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:34.034246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 678/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:36.278789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe0490-088f-4eb0-85cf-8ffc82958f1a None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:36.279384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe0490-088f-4eb0-85cf-8ffc82958f1a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:36.279384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe0490-088f-4eb0-85cf-8ffc82958f1a None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:36.280867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 683/1361] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:36.286075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8178-c861-4d57-aaab-58b1e1e039de None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:36.286523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8178-c861-4d57-aaab-58b1e1e039de None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:36.286614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8178-c861-4d57-aaab-58b1e1e039de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:36.361239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ae4d8178-c861-4d57-aaab-58b1e1e039de None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:36.362345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ae4d8178-c861-4d57-aaab-58b1e1e039de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:36.416327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:36.480012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 679/1362] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:14:36.529009 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe0490-088f-4eb0-85cf-8ffc82958f1a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:36.530678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-26fe0490-088f-4eb0-85cf-8ffc82958f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:36.704507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe0490-088f-4eb0-85cf-8ffc82958f1a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:36.714580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:36.861863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e86780-26ab-42b1-a9c1-c7d715567655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:36.862544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e86780-26ab-42b1-a9c1-c7d715567655 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:36.862939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e86780-26ab-42b1-a9c1-c7d715567655 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:36.863353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e86780-26ab-42b1-a9c1-c7d715567655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:36.865059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-23e86780-26ab-42b1-a9c1-c7d715567655 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:36.958733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e86780-26ab-42b1-a9c1-c7d715567655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:36.960373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e86780-26ab-42b1-a9c1-c7d715567655 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:36.979676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 684/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:39.443178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aade2c0-192c-4c0b-a0c0-10a20c725a11 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:39.444268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aade2c0-192c-4c0b-a0c0-10a20c725a11 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:39.444671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aade2c0-192c-4c0b-a0c0-10a20c725a11 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:39.446790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 680/1364] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:39.452406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678be58e-c043-41ac-8d84-cfa9cd9e82af None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:39.453193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678be58e-c043-41ac-8d84-cfa9cd9e82af None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:39.453610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678be58e-c043-41ac-8d84-cfa9cd9e82af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:39.471755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:39.530719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-678be58e-c043-41ac-8d84-cfa9cd9e82af None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:39.531963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-678be58e-c043-41ac-8d84-cfa9cd9e82af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:39.639189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 685/1365] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:39.681413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f46d-b53f-4783-97f8-e40f16055b25 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:39.683428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f46d-b53f-4783-97f8-e40f16055b25 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:39.683854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f46d-b53f-4783-97f8-e40f16055b25 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:39.685575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 681/1366] 10.222.0.30 () {54 vars in 962 bytes} [Sun Aug 7 12:14:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:39.691219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816612-a1b4-4b9a-b357-ad7d58871c9e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:39.691649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816612-a1b4-4b9a-b357-ad7d58871c9e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:39.692072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816612-a1b4-4b9a-b357-ad7d58871c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:39.774842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d9816612-a1b4-4b9a-b357-ad7d58871c9e None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:39.775989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d9816612-a1b4-4b9a-b357-ad7d58871c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:39.812165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:39.899780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 686/1367] 10.222.0.30 () {58 vars in 1059 bytes} [Sun Aug 7 12:14:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:39.907429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79f46d-b53f-4783-97f8-e40f16055b25 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:39.908941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6e79f46d-b53f-4783-97f8-e40f16055b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:40.073851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79f46d-b53f-4783-97f8-e40f16055b25 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:40.100168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:40.203037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c18ac-6cd0-49a7-89d2-4c798df05f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:40.203685 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c18ac-6cd0-49a7-89d2-4c798df05f45 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:40.203933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c18ac-6cd0-49a7-89d2-4c798df05f45 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:40.204207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c18ac-6cd0-49a7-89d2-4c798df05f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:40.205825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-da3c18ac-6cd0-49a7-89d2-4c798df05f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:40.323328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c18ac-6cd0-49a7-89d2-4c798df05f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:40.325011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c18ac-6cd0-49a7-89d2-4c798df05f45 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:40.351907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 682/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:41.409072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd751b45-e880-4fee-b667-efb383c8600d None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:41.409693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd751b45-e880-4fee-b667-efb383c8600d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:41.409693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd751b45-e880-4fee-b667-efb383c8600d None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:41.411152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 687/1369] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:41.416546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9f1b2-0af9-4329-a103-d95a59907a48 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:41.416948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9f1b2-0af9-4329-a103-d95a59907a48 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:41.417137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9f1b2-0af9-4329-a103-d95a59907a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:41.450404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:41.484270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e9f9f1b2-0af9-4329-a103-d95a59907a48 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:41.485480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e9f9f1b2-0af9-4329-a103-d95a59907a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:41.585848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 683/1370] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:14:41.930967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075d3a8-9534-4668-af67-8cf15669926d None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:41.930967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075d3a8-9534-4668-af67-8cf15669926d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:41.930967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075d3a8-9534-4668-af67-8cf15669926d None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:41.933957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 688/1371] 10.222.0.30 () {54 vars in 964 bytes} [Sun Aug 7 12:14:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:41.939302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b39f2-e0e0-4ea1-ac40-b2c79b38d36c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:41.939701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b39f2-e0e0-4ea1-ac40-b2c79b38d36c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:41.940079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b39f2-e0e0-4ea1-ac40-b2c79b38d36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:42.018439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e39b39f2-e0e0-4ea1-ac40-b2c79b38d36c None None] MFA Rules not processed for user `e0792940b93b4c1998be971f6143a514`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:42.019820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e39b39f2-e0e0-4ea1-ac40-b2c79b38d36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:42.071034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:42.101671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 684/1372] 10.222.0.30 () {58 vars in 1061 bytes} [Sun Aug 7 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:42.110660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075d3a8-9534-4668-af67-8cf15669926d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:42.112407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d075d3a8-9534-4668-af67-8cf15669926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:42.227353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075d3a8-9534-4668-af67-8cf15669926d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:42.266917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:42.302606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f9733-49e1-40b9-ab75-21f192301391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:42.303411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f9733-49e1-40b9-ab75-21f192301391 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:42.303816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f9733-49e1-40b9-ab75-21f192301391 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:42.304226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f9733-49e1-40b9-ab75-21f192301391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:42.306075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0a9f9733-49e1-40b9-ab75-21f192301391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:42.425584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f9733-49e1-40b9-ab75-21f192301391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:42.427458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f9733-49e1-40b9-ab75-21f192301391 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:42.447677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 689/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:14:42.678707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3fa82-fc48-4d44-95ee-9687c99c9d7c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:42.679212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3fa82-fc48-4d44-95ee-9687c99c9d7c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:42.679615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3fa82-fc48-4d44-95ee-9687c99c9d7c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:42.680708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 685/1374] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:42.688384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313e39b-01be-4e03-9b30-054ad5166e6d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:42.688545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313e39b-01be-4e03-9b30-054ad5166e6d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:42.688836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313e39b-01be-4e03-9b30-054ad5166e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:42.762167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e313e39b-01be-4e03-9b30-054ad5166e6d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:42.763859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e313e39b-01be-4e03-9b30-054ad5166e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:42.789452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:42.895034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 690/1375] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:42.919155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3fa82-fc48-4d44-95ee-9687c99c9d7c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:42.920665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3dc3fa82-fc48-4d44-95ee-9687c99c9d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:43.065775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3fa82-fc48-4d44-95ee-9687c99c9d7c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:43.126148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:43.189214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a6b8d-f61c-46c5-80c3-5f43cfdb0a0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:43.189818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a6b8d-f61c-46c5-80c3-5f43cfdb0a0b service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:43.190125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a6b8d-f61c-46c5-80c3-5f43cfdb0a0b service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:43.190399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a6b8d-f61c-46c5-80c3-5f43cfdb0a0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:43.192026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cc0a6b8d-f61c-46c5-80c3-5f43cfdb0a0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:43.284831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a6b8d-f61c-46c5-80c3-5f43cfdb0a0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:43.286416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a6b8d-f61c-46c5-80c3-5f43cfdb0a0b service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:43.304316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 686/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:45.465338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f5581-a5cb-4ded-a3d2-584f8de1c56f None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:45.465856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f5581-a5cb-4ded-a3d2-584f8de1c56f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:45.465856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f5581-a5cb-4ded-a3d2-584f8de1c56f None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:45.467282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 691/1377] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:45.472430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb789-31cf-4243-970c-f27d1f26a6ae None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:45.472826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb789-31cf-4243-970c-f27d1f26a6ae None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:45.472978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb789-31cf-4243-970c-f27d1f26a6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:45.547836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a1deb789-31cf-4243-970c-f27d1f26a6ae None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:45.549042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a1deb789-31cf-4243-970c-f27d1f26a6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:45.556646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:45.668161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 687/1378] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:45.712610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f5581-a5cb-4ded-a3d2-584f8de1c56f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:45.714452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-429f5581-a5cb-4ded-a3d2-584f8de1c56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:45.860962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f5581-a5cb-4ded-a3d2-584f8de1c56f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:45.864572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:45.990265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd9c70-0b65-4e27-8b10-574c9237a4c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:45.990897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd9c70-0b65-4e27-8b10-574c9237a4c6 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:45.991202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd9c70-0b65-4e27-8b10-574c9237a4c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:45.991474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd9c70-0b65-4e27-8b10-574c9237a4c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:45.993886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-38bd9c70-0b65-4e27-8b10-574c9237a4c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:46.103663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd9c70-0b65-4e27-8b10-574c9237a4c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:46.105646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd9c70-0b65-4e27-8b10-574c9237a4c6 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:46.133089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 692/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:48.327589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a23ab-f3aa-4c07-b8ce-bd33e3bb1bb6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:48.328129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a23ab-f3aa-4c07-b8ce-bd33e3bb1bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:48.328129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a23ab-f3aa-4c07-b8ce-bd33e3bb1bb6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:48.329638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 688/1380] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:48.334597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb1a7e-ffd1-4237-96e1-92f964863145 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:48.334989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb1a7e-ffd1-4237-96e1-92f964863145 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:48.335203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb1a7e-ffd1-4237-96e1-92f964863145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:48.347243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:48.406792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-efdb1a7e-ffd1-4237-96e1-92f964863145 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:48.408137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-efdb1a7e-ffd1-4237-96e1-92f964863145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:48.528189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 693/1381] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:48.573003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a23ab-f3aa-4c07-b8ce-bd33e3bb1bb6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:48.575036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-333a23ab-f3aa-4c07-b8ce-bd33e3bb1bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:48.735092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:48.742928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a23ab-f3aa-4c07-b8ce-bd33e3bb1bb6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:48.867180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247eebae-9ee0-4e96-adf2-c90d0d4325c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:48.868058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eebae-9ee0-4e96-adf2-c90d0d4325c6 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:48.868345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eebae-9ee0-4e96-adf2-c90d0d4325c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:48.868625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eebae-9ee0-4e96-adf2-c90d0d4325c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:48.870226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-247eebae-9ee0-4e96-adf2-c90d0d4325c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:48.981689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247eebae-9ee0-4e96-adf2-c90d0d4325c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:48.983261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247eebae-9ee0-4e96-adf2-c90d0d4325c6 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:48.994817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:49.003718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 689/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:51.232718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee60af-eada-4b15-81c4-bb02e4b4a52a None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:51.233384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee60af-eada-4b15-81c4-bb02e4b4a52a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:51.233384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee60af-eada-4b15-81c4-bb02e4b4a52a None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:51.234675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 694/1383] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:51.239795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eb320-303a-457a-b753-2277c0ae4a22 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:51.240243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eb320-303a-457a-b753-2277c0ae4a22 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:51.240346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eb320-303a-457a-b753-2277c0ae4a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:51.314577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ab3eb320-303a-457a-b753-2277c0ae4a22 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:51.316236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ab3eb320-303a-457a-b753-2277c0ae4a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:51.437542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 690/1384] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:51.479810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee60af-eada-4b15-81c4-bb02e4b4a52a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:51.481438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b2ee60af-eada-4b15-81c4-bb02e4b4a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:51.571792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:51.642731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee60af-eada-4b15-81c4-bb02e4b4a52a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:51.773156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac20b5-2c3f-4420-abbb-2f5815e625f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:51.773821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac20b5-2c3f-4420-abbb-2f5815e625f2 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:51.774247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac20b5-2c3f-4420-abbb-2f5815e625f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:51.774526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac20b5-2c3f-4420-abbb-2f5815e625f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:51.776176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-29ac20b5-2c3f-4420-abbb-2f5815e625f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:51.827688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:51.869574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac20b5-2c3f-4420-abbb-2f5815e625f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:51.871237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac20b5-2c3f-4420-abbb-2f5815e625f2 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:51.890931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 695/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:14:54.173411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9cfaf-5a80-4cbf-b5f0-2ac7873e81e7 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:54.174109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9cfaf-5a80-4cbf-b5f0-2ac7873e81e7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:54.174109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9cfaf-5a80-4cbf-b5f0-2ac7873e81e7 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:54.175359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 691/1386] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:54.181107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85da8d8-15f2-4fab-b894-ba0d93aa63e4 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:54.181533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85da8d8-15f2-4fab-b894-ba0d93aa63e4 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:54.181813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85da8d8-15f2-4fab-b894-ba0d93aa63e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:54.250634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a85da8d8-15f2-4fab-b894-ba0d93aa63e4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:54.251861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a85da8d8-15f2-4fab-b894-ba0d93aa63e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:54.348576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 696/1387] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:14:54.396167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9cfaf-5a80-4cbf-b5f0-2ac7873e81e7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:54.397828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b3e9cfaf-5a80-4cbf-b5f0-2ac7873e81e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:54.484463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:54.559468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9cfaf-5a80-4cbf-b5f0-2ac7873e81e7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:54.682792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fc422-862d-4345-b2f4-53ba3de0dc8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:54.683424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fc422-862d-4345-b2f4-53ba3de0dc8f service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:54.683697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fc422-862d-4345-b2f4-53ba3de0dc8f service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:54.683966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fc422-862d-4345-b2f4-53ba3de0dc8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:54.685637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-254fc422-862d-4345-b2f4-53ba3de0dc8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:54.723177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:54.780258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fc422-862d-4345-b2f4-53ba3de0dc8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:54.781908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fc422-862d-4345-b2f4-53ba3de0dc8f service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:54.800907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 692/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:56.915845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fef817-dafb-4175-8c0d-c7ef6834808e None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:56.916469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fef817-dafb-4175-8c0d-c7ef6834808e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:56.916469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fef817-dafb-4175-8c0d-c7ef6834808e None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:56.917775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 697/1389] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:14:56.923648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cacb9-9c1a-4d16-88c4-8df61922ec9d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:56.924245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cacb9-9c1a-4d16-88c4-8df61922ec9d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:56.924245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cacb9-9c1a-4d16-88c4-8df61922ec9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:56.990286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7c9cacb9-9c1a-4d16-88c4-8df61922ec9d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:56.991941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7c9cacb9-9c1a-4d16-88c4-8df61922ec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:57.089636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 693/1390] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:57.134282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fef817-dafb-4175-8c0d-c7ef6834808e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:57.135839 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-97fef817-dafb-4175-8c0d-c7ef6834808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:57.166338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:57.284020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fef817-dafb-4175-8c0d-c7ef6834808e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:57.387980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:57.399377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d280869-7c21-4487-abd8-9b5a111d849d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:57.399990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280869-7c21-4487-abd8-9b5a111d849d service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:57.400273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280869-7c21-4487-abd8-9b5a111d849d service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:57.400546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280869-7c21-4487-abd8-9b5a111d849d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:57.402153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5d280869-7c21-4487-abd8-9b5a111d849d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:57.492667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d280869-7c21-4487-abd8-9b5a111d849d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:57.494278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d280869-7c21-4487-abd8-9b5a111d849d service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:57.512333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 698/1391] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:14:57.834430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb919c-d481-4ecb-b8c3-ef69ed11c562 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:57.834725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb919c-d481-4ecb-b8c3-ef69ed11c562 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:57.835648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb919c-d481-4ecb-b8c3-ef69ed11c562 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:57.837442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 694/1392] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:57.843028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61f63-8dbb-4738-bc07-bce1408c794b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:57.843425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61f63-8dbb-4738-bc07-bce1408c794b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:57.843761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61f63-8dbb-4738-bc07-bce1408c794b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:57.911486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-b5f61f63-8dbb-4738-bc07-bce1408c794b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:57.912720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b5f61f63-8dbb-4738-bc07-bce1408c794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:57.974305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:58.013423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 699/1393] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:58.068382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd78be-1b2a-4cbe-8a8a-f52b6690274c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:58.068850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd78be-1b2a-4cbe-8a8a-f52b6690274c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:58.068946 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd78be-1b2a-4cbe-8a8a-f52b6690274c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:58.071015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 695/1394] 10.222.0.30 () {54 vars in 966 bytes} [Sun Aug 7 12:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:58.076285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f9bdc-8f32-4ab1-aede-51f33e8c8af1 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:58.076579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f9bdc-8f32-4ab1-aede-51f33e8c8af1 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:58.076882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f9bdc-8f32-4ab1-aede-51f33e8c8af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:58.156838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0d4f9bdc-8f32-4ab1-aede-51f33e8c8af1 None None] MFA Rules not processed for user `6b5ee9e4388d4bbabc7f1e9e3ca50b67`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:58.158411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0d4f9bdc-8f32-4ab1-aede-51f33e8c8af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:58.235402 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 700/1395] 10.222.0.30 () {58 vars in 1063 bytes} [Sun Aug 7 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:14:58.242887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd78be-1b2a-4cbe-8a8a-f52b6690274c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:58.244429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a0dd78be-1b2a-4cbe-8a8a-f52b6690274c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:58.258857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:58.349309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd78be-1b2a-4cbe-8a8a-f52b6690274c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:58.425430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513e361-f30b-441f-ad1b-124916b18bfa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b5ee9e4388d4bbabc7f1e9e3ca50b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:14:58.426041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e361-f30b-441f-ad1b-124916b18bfa service glance] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:58.426323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e361-f30b-441f-ad1b-124916b18bfa service glance] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:58.426597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e361-f30b-441f-ad1b-124916b18bfa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:58.428252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b513e361-f30b-441f-ad1b-124916b18bfa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:58.499090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:58.540975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e361-f30b-441f-ad1b-124916b18bfa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:14:58.542405 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e361-f30b-441f-ad1b-124916b18bfa service glance] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:14:58.561905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 696/1396] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:14:59.450907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbf2f7-99f7-40b3-8067-b1ecc5fdc87a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:59.451447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbf2f7-99f7-40b3-8067-b1ecc5fdc87a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:59.451545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbf2f7-99f7-40b3-8067-b1ecc5fdc87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:59.520643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-5ebbf2f7-99f7-40b3-8067-b1ecc5fdc87a None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:59.521787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5ebbf2f7-99f7-40b3-8067-b1ecc5fdc87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:59.531474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:59.611441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 701/1397] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:14:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:59.659409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711fd69-16a3-4c3f-88c7-140a47ce2e9f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:59.659878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711fd69-16a3-4c3f-88c7-140a47ce2e9f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:59.659999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711fd69-16a3-4c3f-88c7-140a47ce2e9f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:59.660133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8521-ff33-4949-966f-776d62b844ed None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:59.660287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8521-ff33-4949-966f-776d62b844ed None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:59.660566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8521-ff33-4949-966f-776d62b844ed None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:59.661392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 697/1398] 10.222.0.30 () {54 vars in 943 bytes} [Sun Aug 7 12:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:14:59.661899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 702/1399] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:14:59.666942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01026-27f1-4c77-b9c7-499a9b9a3a1f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:59.667242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01026-27f1-4c77-b9c7-499a9b9a3a1f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:59.667521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01026-27f1-4c77-b9c7-499a9b9a3a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:59.667982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29d6f4-5aa2-4b2d-982a-11b136e3d6a7 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:14:59.668267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29d6f4-5aa2-4b2d-982a-11b136e3d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:14:59.668543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29d6f4-5aa2-4b2d-982a-11b136e3d6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:14:59.735753 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8dc01026-27f1-4c77-b9c7-499a9b9a3a1f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:59.736960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8dc01026-27f1-4c77-b9c7-499a9b9a3a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:59.746859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4e29d6f4-5aa2-4b2d-982a-11b136e3d6a7 None None] MFA Rules not processed for user `a66a21436030409484b17f803d0e5903`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:14:59.748078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4e29d6f4-5aa2-4b2d-982a-11b136e3d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:59.835521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:59.837422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 698/1400] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:14:59.857039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 703/1401] 10.222.0.30 () {58 vars in 1040 bytes} [Sun Aug 7 12:14:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:14:59.866224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc01026-27f1-4c77-b9c7-499a9b9a3a1f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:59.867649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8dc01026-27f1-4c77-b9c7-499a9b9a3a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:59.881027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29d6f4-5aa2-4b2d-982a-11b136e3d6a7 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:14:59.882718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4e29d6f4-5aa2-4b2d-982a-11b136e3d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:14:59.895696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:14:59.971908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc01026-27f1-4c77-b9c7-499a9b9a3a1f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:14:59.980938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29d6f4-5aa2-4b2d-982a-11b136e3d6a7 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:00.075630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119101e4-b9de-420c-b8dd-e7e22b5b3f06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:00.076225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119101e4-b9de-420c-b8dd-e7e22b5b3f06 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:00.076500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119101e4-b9de-420c-b8dd-e7e22b5b3f06 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:00.076775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119101e4-b9de-420c-b8dd-e7e22b5b3f06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:00.078382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-119101e4-b9de-420c-b8dd-e7e22b5b3f06 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:00.091181 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:00.101200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08af49-5a65-49d1-92ac-7046423e6954 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:00.101809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08af49-5a65-49d1-92ac-7046423e6954 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:00.102177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08af49-5a65-49d1-92ac-7046423e6954 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:00.102454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08af49-5a65-49d1-92ac-7046423e6954 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:00.106914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3f08af49-5a65-49d1-92ac-7046423e6954 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:00.123089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:00.167281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119101e4-b9de-420c-b8dd-e7e22b5b3f06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:00.168891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119101e4-b9de-420c-b8dd-e7e22b5b3f06 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:00.177001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 699/1402] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:14:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:15:00.201272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08af49-5a65-49d1-92ac-7046423e6954 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:00.202883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08af49-5a65-49d1-92ac-7046423e6954 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:00.221491 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 704/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:15:02.590308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172e111-e358-4047-8ffa-f2324f077e2a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:02.590882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172e111-e358-4047-8ffa-f2324f077e2a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:02.590882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172e111-e358-4047-8ffa-f2324f077e2a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:02.592480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 700/1404] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:02.598160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad1f62-0d0c-4d19-b19d-15ff5a953acb None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:02.598576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad1f62-0d0c-4d19-b19d-15ff5a953acb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:02.598756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad1f62-0d0c-4d19-b19d-15ff5a953acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:02.675035 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-20ad1f62-0d0c-4d19-b19d-15ff5a953acb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:02.676338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-20ad1f62-0d0c-4d19-b19d-15ff5a953acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:02.743920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:02.788668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 705/1405] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:02.837202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172e111-e358-4047-8ffa-f2324f077e2a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:02.838920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4172e111-e358-4047-8ffa-f2324f077e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:03.001639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172e111-e358-4047-8ffa-f2324f077e2a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:03.042072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:03.132923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d65c5-febf-46ca-b2fa-880f052d815c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:03.133533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d65c5-febf-46ca-b2fa-880f052d815c service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:03.133811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d65c5-febf-46ca-b2fa-880f052d815c service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:03.134081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d65c5-febf-46ca-b2fa-880f052d815c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:03.136856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-be0d65c5-febf-46ca-b2fa-880f052d815c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:03.237020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d65c5-febf-46ca-b2fa-880f052d815c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:03.238615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d65c5-febf-46ca-b2fa-880f052d815c service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:03.259836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 701/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:05.445747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1cabf-9bc7-4573-bd95-71ac7e361d55 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:05.446331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1cabf-9bc7-4573-bd95-71ac7e361d55 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:05.446331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1cabf-9bc7-4573-bd95-71ac7e361d55 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:05.447826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 706/1407] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:05.452797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296575c-2a43-48b0-a4f1-3e141e8eaca8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:05.453206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296575c-2a43-48b0-a4f1-3e141e8eaca8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:05.453382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296575c-2a43-48b0-a4f1-3e141e8eaca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:05.504796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:05.525360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7296575c-2a43-48b0-a4f1-3e141e8eaca8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:05.526621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7296575c-2a43-48b0-a4f1-3e141e8eaca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:05.763711 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 702/1408] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:05.810905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1cabf-9bc7-4573-bd95-71ac7e361d55 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:05.812564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-15a1cabf-9bc7-4573-bd95-71ac7e361d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:05.977989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1cabf-9bc7-4573-bd95-71ac7e361d55 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:06.036482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:06.105968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36e6cf-71a1-46ec-b88a-ff596586ad41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:06.106570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36e6cf-71a1-46ec-b88a-ff596586ad41 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:06.106845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36e6cf-71a1-46ec-b88a-ff596586ad41 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:06.107167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36e6cf-71a1-46ec-b88a-ff596586ad41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:06.110808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3a36e6cf-71a1-46ec-b88a-ff596586ad41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:06.216377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36e6cf-71a1-46ec-b88a-ff596586ad41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:06.218023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36e6cf-71a1-46ec-b88a-ff596586ad41 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:06.239048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 707/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:15.761779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=47 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:16.236450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=75 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:23.449821 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b3bde-866c-4952-afac-9a55550c8f8a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:23.450409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b3bde-866c-4952-afac-9a55550c8f8a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:23.450409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b3bde-866c-4952-afac-9a55550c8f8a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:23.451737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 703/1410] 10.222.0.30 () {54 vars in 918 bytes} [Sun Aug 7 12:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:23.456765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64700f-711a-43ef-bfaf-f5ba8aba9fec None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:23.457173 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64700f-711a-43ef-bfaf-f5ba8aba9fec None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:23.457310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64700f-711a-43ef-bfaf-f5ba8aba9fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:23.533257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bb64700f-711a-43ef-bfaf-f5ba8aba9fec None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:23.534457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bb64700f-711a-43ef-bfaf-f5ba8aba9fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:23.637856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 708/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Sun Aug 7 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:23.653553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef1667-91a9-4cc0-9e66-00bd194550f0 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:23.653835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef1667-91a9-4cc0-9e66-00bd194550f0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:23.654105 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef1667-91a9-4cc0-9e66-00bd194550f0 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:23.655529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 704/1412] 10.222.0.30 () {54 vars in 918 bytes} [Sun Aug 7 12:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:23.660199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edb6d0-5444-40c1-98f9-7b7b6a5eedb0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:23.660487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edb6d0-5444-40c1-98f9-7b7b6a5eedb0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:23.660758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edb6d0-5444-40c1-98f9-7b7b6a5eedb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:23.733929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:23.738149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-95edb6d0-5444-40c1-98f9-7b7b6a5eedb0 None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:23.739407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-95edb6d0-5444-40c1-98f9-7b7b6a5eedb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:23.856109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 709/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Sun Aug 7 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:26.378729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ec257-a200-4986-a3cc-04d31d450fdc None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:26.379288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ec257-a200-4986-a3cc-04d31d450fdc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:26.379288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ec257-a200-4986-a3cc-04d31d450fdc None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:26.380649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:26.386660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ddce-1d01-4eb4-b863-5e5a4c8b8b34 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:26.387103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ddce-1d01-4eb4-b863-5e5a4c8b8b34 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:26.387298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ddce-1d01-4eb4-b863-5e5a4c8b8b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:26.452082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-9f62ddce-1d01-4eb4-b863-5e5a4c8b8b34 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:26.453284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9f62ddce-1d01-4eb4-b863-5e5a4c8b8b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:26.507679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:26.549316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:26.592898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ec257-a200-4986-a3cc-04d31d450fdc None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:26.594417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-543ec257-a200-4986-a3cc-04d31d450fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:26.740159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ec257-a200-4986-a3cc-04d31d450fdc None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:26.815011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:26.855375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ab012-b5a0-44c4-96f2-b865dfdc658c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:26.855971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ab012-b5a0-44c4-96f2-b865dfdc658c service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:26.856244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ab012-b5a0-44c4-96f2-b865dfdc658c service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:26.856510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ab012-b5a0-44c4-96f2-b865dfdc658c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:26.858076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-380ab012-b5a0-44c4-96f2-b865dfdc658c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:26.947382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ab012-b5a0-44c4-96f2-b865dfdc658c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:26.948965 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ab012-b5a0-44c4-96f2-b865dfdc658c service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:26.967042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 706/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:27.876126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0760c-d0f8-4acd-aef7-0ade7d85fd67 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:27.876714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0760c-d0f8-4acd-aef7-0ade7d85fd67 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:27.876714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0760c-d0f8-4acd-aef7-0ade7d85fd67 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:27.878114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:27.883912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae2afe-5275-4427-bf1e-775febb14efe None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:27.884383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae2afe-5275-4427-bf1e-775febb14efe None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:27.884480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae2afe-5275-4427-bf1e-775febb14efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:27.959252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e2ae2afe-5275-4427-bf1e-775febb14efe None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:27.960538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e2ae2afe-5275-4427-bf1e-775febb14efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:27.986703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:28.055755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:28.205572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0760c-d0f8-4acd-aef7-0ade7d85fd67 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:28.207047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-24b0760c-d0f8-4acd-aef7-0ade7d85fd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:28.353644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0760c-d0f8-4acd-aef7-0ade7d85fd67 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:28.393587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:28.466744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca76fd6-73f8-4559-a5fb-915336a64f8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:28.467274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca76fd6-73f8-4559-a5fb-915336a64f8c service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:28.467560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca76fd6-73f8-4559-a5fb-915336a64f8c service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:28.467830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca76fd6-73f8-4559-a5fb-915336a64f8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:28.469429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0ca76fd6-73f8-4559-a5fb-915336a64f8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:28.558761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca76fd6-73f8-4559-a5fb-915336a64f8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:28.560411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca76fd6-73f8-4559-a5fb-915336a64f8c service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:28.578172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 712/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:29.465451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43c3a5-7348-4acd-8974-fd5840102006 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:29.466012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43c3a5-7348-4acd-8974-fd5840102006 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:29.466125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43c3a5-7348-4acd-8974-fd5840102006 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:29.467483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:29.472520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc100c-c59a-4d31-9747-329c26beb67b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:29.472965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc100c-c59a-4d31-9747-329c26beb67b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:29.473076 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc100c-c59a-4d31-9747-329c26beb67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:29.527815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:29.549274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c1bc100c-c59a-4d31-9747-329c26beb67b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:29.550485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c1bc100c-c59a-4d31-9747-329c26beb67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:29.648134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:29.800985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43c3a5-7348-4acd-8974-fd5840102006 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:29.802478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4e43c3a5-7348-4acd-8974-fd5840102006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:29.945754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43c3a5-7348-4acd-8974-fd5840102006 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:29.957503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:30.057530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b925a-39c4-436c-a269-d30fe7b1a0f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:30.058121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b925a-39c4-436c-a269-d30fe7b1a0f6 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:30.058394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b925a-39c4-436c-a269-d30fe7b1a0f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:30.058659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b925a-39c4-436c-a269-d30fe7b1a0f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:30.060256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-632b925a-39c4-436c-a269-d30fe7b1a0f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:30.151720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b925a-39c4-436c-a269-d30fe7b1a0f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:30.153306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b925a-39c4-436c-a269-d30fe7b1a0f6 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:30.171061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 709/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:31.040132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854afb1b-003d-4b61-af67-19a8bb64ff53 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:31.040689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854afb1b-003d-4b61-af67-19a8bb64ff53 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:31.040689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854afb1b-003d-4b61-af67-19a8bb64ff53 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:31.041980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:31.046884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd9e52-9c98-4e74-aa26-a3f29fddea45 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:31.047323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd9e52-9c98-4e74-aa26-a3f29fddea45 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:31.047458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd9e52-9c98-4e74-aa26-a3f29fddea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:31.071431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:31.135108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-23bd9e52-9c98-4e74-aa26-a3f29fddea45 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:31.136330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-23bd9e52-9c98-4e74-aa26-a3f29fddea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:31.230449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:31.380520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854afb1b-003d-4b61-af67-19a8bb64ff53 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:31.382345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-854afb1b-003d-4b61-af67-19a8bb64ff53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:31.493510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:31.529524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854afb1b-003d-4b61-af67-19a8bb64ff53 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:31.640469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36278d38-8f72-42bb-a088-e889d090d92c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:31.640995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36278d38-8f72-42bb-a088-e889d090d92c service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:31.641273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36278d38-8f72-42bb-a088-e889d090d92c service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:31.641544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36278d38-8f72-42bb-a088-e889d090d92c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:31.643182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-36278d38-8f72-42bb-a088-e889d090d92c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:31.720005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:31.733392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36278d38-8f72-42bb-a088-e889d090d92c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:31.734992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36278d38-8f72-42bb-a088-e889d090d92c service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:31.753019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 715/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:32.605015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac698-27e2-427b-a853-8fa044cf408d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:32.605563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac698-27e2-427b-a853-8fa044cf408d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:32.605563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac698-27e2-427b-a853-8fa044cf408d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:32.606884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:32.611680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec5167-9c7a-4287-bcdb-1a5f86f1d9f4 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:32.612097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec5167-9c7a-4287-bcdb-1a5f86f1d9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:32.612225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec5167-9c7a-4287-bcdb-1a5f86f1d9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:32.686866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-40ec5167-9c7a-4287-bcdb-1a5f86f1d9f4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:32.688097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-40ec5167-9c7a-4287-bcdb-1a5f86f1d9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:32.784496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:32.935995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac698-27e2-427b-a853-8fa044cf408d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:32.937514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-261ac698-27e2-427b-a853-8fa044cf408d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:33.022143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:33.083328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac698-27e2-427b-a853-8fa044cf408d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:33.195508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82109bfb-5b75-4f84-be4c-bd6c1974cf1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:33.196020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82109bfb-5b75-4f84-be4c-bd6c1974cf1d service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:33.196300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82109bfb-5b75-4f84-be4c-bd6c1974cf1d service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:33.196574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82109bfb-5b75-4f84-be4c-bd6c1974cf1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:33.198164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-82109bfb-5b75-4f84-be4c-bd6c1974cf1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:33.246030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:33.289023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82109bfb-5b75-4f84-be4c-bd6c1974cf1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:33.290610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82109bfb-5b75-4f84-be4c-bd6c1974cf1d service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:33.308709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 712/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:34.171639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ccf12-b43e-42c5-9140-9810fc2fe353 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:34.172203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ccf12-b43e-42c5-9140-9810fc2fe353 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:34.172203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ccf12-b43e-42c5-9140-9810fc2fe353 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:34.173550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:34.178414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e1c3-2c07-4f53-a5c3-2668454c7903 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:34.178860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e1c3-2c07-4f53-a5c3-2668454c7903 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:34.179593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e1c3-2c07-4f53-a5c3-2668454c7903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:34.255609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f6f6e1c3-2c07-4f53-a5c3-2668454c7903 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:34.256771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f6f6e1c3-2c07-4f53-a5c3-2668454c7903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:34.352062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:34.503434 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ccf12-b43e-42c5-9140-9810fc2fe353 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:34.504975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-de6ccf12-b43e-42c5-9140-9810fc2fe353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:34.554798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:34.651551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ccf12-b43e-42c5-9140-9810fc2fe353 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:34.763057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fa2c2-36ee-43a9-b70d-66843760061f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:34.763664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa2c2-36ee-43a9-b70d-66843760061f service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:34.763943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa2c2-36ee-43a9-b70d-66843760061f service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:34.764215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fa2c2-36ee-43a9-b70d-66843760061f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:34.765801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7b0fa2c2-36ee-43a9-b70d-66843760061f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:34.773764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:34.980084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fa2c2-36ee-43a9-b70d-66843760061f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:34.981677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fa2c2-36ee-43a9-b70d-66843760061f service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:34.999938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 718/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:35.845428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e0f5-3eea-4d16-8431-f5e97b25db0e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:35.845989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e0f5-3eea-4d16-8431-f5e97b25db0e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:35.845989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e0f5-3eea-4d16-8431-f5e97b25db0e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:35.847346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:35.852852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2a84-c131-4d08-80b5-67e0cc5d816e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:35.853301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2a84-c131-4d08-80b5-67e0cc5d816e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:35.853409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2a84-c131-4d08-80b5-67e0cc5d816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:35.929681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-164e2a84-c131-4d08-80b5-67e0cc5d816e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:35.930881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-164e2a84-c131-4d08-80b5-67e0cc5d816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:35.997400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:36.026487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:36.178849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099e0f5-3eea-4d16-8431-f5e97b25db0e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:36.180507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b099e0f5-3eea-4d16-8431-f5e97b25db0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:36.198363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:36.326379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099e0f5-3eea-4d16-8431-f5e97b25db0e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:36.413464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:36.438866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0075c7-a327-4b2e-84b7-834ad9b2c049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:36.444512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0075c7-a327-4b2e-84b7-834ad9b2c049 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:36.447451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0075c7-a327-4b2e-84b7-834ad9b2c049 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:36.447451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0075c7-a327-4b2e-84b7-834ad9b2c049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:36.447451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ff0075c7-a327-4b2e-84b7-834ad9b2c049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:36.536561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0075c7-a327-4b2e-84b7-834ad9b2c049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:36.538185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0075c7-a327-4b2e-84b7-834ad9b2c049 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:36.556131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 715/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:37.417075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac69bb-2b06-4cff-ae05-5d6469b860bf None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:37.417608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac69bb-2b06-4cff-ae05-5d6469b860bf None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:37.417608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac69bb-2b06-4cff-ae05-5d6469b860bf None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:37.419034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:37.424111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61fe6-a339-41e7-9be0-a67d59834900 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:37.424547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61fe6-a339-41e7-9be0-a67d59834900 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:37.424660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61fe6-a339-41e7-9be0-a67d59834900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:37.504167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-27e61fe6-a339-41e7-9be0-a67d59834900 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:37.505533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-27e61fe6-a339-41e7-9be0-a67d59834900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:37.548305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:37.603160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:15:37.753845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac69bb-2b06-4cff-ae05-5d6469b860bf None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:37.755310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1dac69bb-2b06-4cff-ae05-5d6469b860bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:37.897899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac69bb-2b06-4cff-ae05-5d6469b860bf None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:37.952620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:38.010262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed544ca-a0e3-432a-ad4b-ede6aa496d7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:38.010785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed544ca-a0e3-432a-ad4b-ede6aa496d7c service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:38.011089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed544ca-a0e3-432a-ad4b-ede6aa496d7c service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:38.011365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed544ca-a0e3-432a-ad4b-ede6aa496d7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:38.012951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6ed544ca-a0e3-432a-ad4b-ede6aa496d7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:38.100903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed544ca-a0e3-432a-ad4b-ede6aa496d7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:38.102512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed544ca-a0e3-432a-ad4b-ede6aa496d7c service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:38.121849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 721/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:38.982374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151efd3-9cd4-4fe5-b6fb-25318b628545 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:38.982374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151efd3-9cd4-4fe5-b6fb-25318b628545 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:38.982826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151efd3-9cd4-4fe5-b6fb-25318b628545 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:38.983877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:38.988712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f91ab-b001-498f-ac6e-efb52732d112 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:38.989111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f91ab-b001-498f-ac6e-efb52732d112 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:38.989265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f91ab-b001-498f-ac6e-efb52732d112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:39.065735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-166f91ab-b001-498f-ac6e-efb52732d112 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:39.066867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-166f91ab-b001-498f-ac6e-efb52732d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:39.070001 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:39.166210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:39.317285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151efd3-9cd4-4fe5-b6fb-25318b628545 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:39.318987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0151efd3-9cd4-4fe5-b6fb-25318b628545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:39.465468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151efd3-9cd4-4fe5-b6fb-25318b628545 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:39.492218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:39.578684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84735f1a-0e51-4c7a-b8c8-669561511dd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:39.579265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735f1a-0e51-4c7a-b8c8-669561511dd2 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:39.579543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735f1a-0e51-4c7a-b8c8-669561511dd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:39.579814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735f1a-0e51-4c7a-b8c8-669561511dd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:39.581600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-84735f1a-0e51-4c7a-b8c8-669561511dd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:39.671006 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84735f1a-0e51-4c7a-b8c8-669561511dd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:39.672613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84735f1a-0e51-4c7a-b8c8-669561511dd2 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:39.690774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 718/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:41.690094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88ed16-abd6-4c00-86ae-7adf35a9ad62 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:41.690642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88ed16-abd6-4c00-86ae-7adf35a9ad62 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:41.690642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88ed16-abd6-4c00-86ae-7adf35a9ad62 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:41.692006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:41.697284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815618-9aad-40bb-bc9c-8ce87ce67a08 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:41.697748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815618-9aad-40bb-bc9c-8ce87ce67a08 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:41.697855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815618-9aad-40bb-bc9c-8ce87ce67a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:41.738184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:41.763670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9f815618-9aad-40bb-bc9c-8ce87ce67a08 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:41.764901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9f815618-9aad-40bb-bc9c-8ce87ce67a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:41.861630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:41.900073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88ed16-abd6-4c00-86ae-7adf35a9ad62 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:41.901595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9f88ed16-abd6-4c00-86ae-7adf35a9ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:42.004699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88ed16-abd6-4c00-86ae-7adf35a9ad62 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:42.037762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:42.077915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173959e-87b8-4d3e-8760-817a0021fa3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:42.078521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173959e-87b8-4d3e-8760-817a0021fa3e service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:42.078817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173959e-87b8-4d3e-8760-817a0021fa3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:42.079119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173959e-87b8-4d3e-8760-817a0021fa3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:42.080722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6173959e-87b8-4d3e-8760-817a0021fa3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:42.189705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173959e-87b8-4d3e-8760-817a0021fa3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:42.191202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173959e-87b8-4d3e-8760-817a0021fa3e service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:42.209213 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 724/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:15:44.345358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8850034-f6c3-46df-a4b6-607389a2ef60 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:44.345920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8850034-f6c3-46df-a4b6-607389a2ef60 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:44.345920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8850034-f6c3-46df-a4b6-607389a2ef60 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:44.347266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:44.352463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d80186-1de5-4e54-9da2-f019428897aa None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:44.352922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d80186-1de5-4e54-9da2-f019428897aa None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:44.353032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d80186-1de5-4e54-9da2-f019428897aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:44.418305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c9d80186-1de5-4e54-9da2-f019428897aa None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:44.419529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c9d80186-1de5-4e54-9da2-f019428897aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:44.433102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:44.516468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:44.554968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8850034-f6c3-46df-a4b6-607389a2ef60 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:44.556515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c8850034-f6c3-46df-a4b6-607389a2ef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:44.659520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8850034-f6c3-46df-a4b6-607389a2ef60 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:44.671649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:44.732793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c37e7b-8ba9-4a0d-9dac-45e73e2e8f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:44.733394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c37e7b-8ba9-4a0d-9dac-45e73e2e8f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:44.733673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c37e7b-8ba9-4a0d-9dac-45e73e2e8f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:44.733947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c37e7b-8ba9-4a0d-9dac-45e73e2e8f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:44.735606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-30c37e7b-8ba9-4a0d-9dac-45e73e2e8f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:44.844694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c37e7b-8ba9-4a0d-9dac-45e73e2e8f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:44.846128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c37e7b-8ba9-4a0d-9dac-45e73e2e8f9e service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:44.863911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 721/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:47.229194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6b14-ad0e-41f7-bb1b-7d0d3ce7276d None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:47.229771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6b14-ad0e-41f7-bb1b-7d0d3ce7276d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:47.229771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6b14-ad0e-41f7-bb1b-7d0d3ce7276d None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:47.231105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:47.236331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9e31-1749-4eb1-b3c9-8d610ee65632 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:47.236757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9e31-1749-4eb1-b3c9-8d610ee65632 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:47.236871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9e31-1749-4eb1-b3c9-8d610ee65632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:47.285446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:47.303646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-37fa9e31-1749-4eb1-b3c9-8d610ee65632 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:47.307178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-37fa9e31-1749-4eb1-b3c9-8d610ee65632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:47.402231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:15:47.442376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa6b14-ad0e-41f7-bb1b-7d0d3ce7276d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:47.444179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8aaa6b14-ad0e-41f7-bb1b-7d0d3ce7276d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:47.546450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa6b14-ad0e-41f7-bb1b-7d0d3ce7276d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:47.585462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:47.618807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de154052-4964-4678-a26c-e6ba33e6c539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:47.619433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de154052-4964-4678-a26c-e6ba33e6c539 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:47.619707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de154052-4964-4678-a26c-e6ba33e6c539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:47.619974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de154052-4964-4678-a26c-e6ba33e6c539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:47.621541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-de154052-4964-4678-a26c-e6ba33e6c539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:47.727608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de154052-4964-4678-a26c-e6ba33e6c539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:47.728958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de154052-4964-4678-a26c-e6ba33e6c539 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:47.746895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 727/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:49.914807 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd638-f97e-4dd7-a8f4-cd053710c019 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:49.915407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd638-f97e-4dd7-a8f4-cd053710c019 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:49.915407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd638-f97e-4dd7-a8f4-cd053710c019 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:49.916734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:49.921963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa460c1-e730-43f8-9b1b-bd8e0596608b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:49.922369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa460c1-e730-43f8-9b1b-bd8e0596608b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:49.922509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa460c1-e730-43f8-9b1b-bd8e0596608b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:49.996700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1fa460c1-e730-43f8-9b1b-bd8e0596608b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:49.997892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1fa460c1-e730-43f8-9b1b-bd8e0596608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:50.017944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:50.091924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:50.132588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacd638-f97e-4dd7-a8f4-cd053710c019 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:50.134117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cdacd638-f97e-4dd7-a8f4-cd053710c019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:50.236387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacd638-f97e-4dd7-a8f4-cd053710c019 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:50.256413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:50.309780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38cdb2-a46b-4964-8640-08c5906bf839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:50.310377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cdb2-a46b-4964-8640-08c5906bf839 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:50.310654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cdb2-a46b-4964-8640-08c5906bf839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:50.310924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cdb2-a46b-4964-8640-08c5906bf839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:50.312535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2c38cdb2-a46b-4964-8640-08c5906bf839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:50.419842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38cdb2-a46b-4964-8640-08c5906bf839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:50.421256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38cdb2-a46b-4964-8640-08c5906bf839 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:50.438882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 724/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:52.570892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d70c-333e-4fdd-808a-23bbc4a949cc None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:52.571496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d70c-333e-4fdd-808a-23bbc4a949cc None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:52.571496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d70c-333e-4fdd-808a-23bbc4a949cc None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:52.572840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:52.578116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0586-e071-4d1a-8985-c4b51009e24c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:52.578543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0586-e071-4d1a-8985-c4b51009e24c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:52.578652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0586-e071-4d1a-8985-c4b51009e24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:52.641411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:52.643958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-05ef0586-e071-4d1a-8985-c4b51009e24c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:52.645155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-05ef0586-e071-4d1a-8985-c4b51009e24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:52.741962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:52.781086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98d70c-333e-4fdd-808a-23bbc4a949cc None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:52.782959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bd98d70c-333e-4fdd-808a-23bbc4a949cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:52.885185 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98d70c-333e-4fdd-808a-23bbc4a949cc None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:52.933227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:52.957822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27f2f7-7a22-4bb6-8c70-c9576817a69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:52.959053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27f2f7-7a22-4bb6-8c70-c9576817a69a service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:52.959330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27f2f7-7a22-4bb6-8c70-c9576817a69a service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:52.959597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27f2f7-7a22-4bb6-8c70-c9576817a69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:52.961164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4e27f2f7-7a22-4bb6-8c70-c9576817a69a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:53.069431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27f2f7-7a22-4bb6-8c70-c9576817a69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:53.070872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27f2f7-7a22-4bb6-8c70-c9576817a69a service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:53.091119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 730/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:55.161453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135e52-4b0f-4205-aa3b-d02cf15f515b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:55.162120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135e52-4b0f-4205-aa3b-d02cf15f515b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:55.162120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135e52-4b0f-4205-aa3b-d02cf15f515b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:55.163400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:55.168677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea610-7ed1-42a2-ad2c-40f45d065159 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:55.169093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea610-7ed1-42a2-ad2c-40f45d065159 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:55.169206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea610-7ed1-42a2-ad2c-40f45d065159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:55.234463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6d9ea610-7ed1-42a2-ad2c-40f45d065159 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:55.235688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6d9ea610-7ed1-42a2-ad2c-40f45d065159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:55.265327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:55.332939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:55.372358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135e52-4b0f-4205-aa3b-d02cf15f515b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:55.373931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bd135e52-4b0f-4205-aa3b-d02cf15f515b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:55.479760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135e52-4b0f-4205-aa3b-d02cf15f515b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:55.509933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:55.555778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba8367-607a-4150-9691-2dbf7a6bc229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:55.556379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba8367-607a-4150-9691-2dbf7a6bc229 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:55.556656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba8367-607a-4150-9691-2dbf7a6bc229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:55.556928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba8367-607a-4150-9691-2dbf7a6bc229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:55.558517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6ba8367-607a-4150-9691-2dbf7a6bc229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:55.665609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba8367-607a-4150-9691-2dbf7a6bc229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:55.667122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba8367-607a-4150-9691-2dbf7a6bc229 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:55.685320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 727/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:15:57.714438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083a3b4-304f-4483-b9a8-83c6ce2aa37d None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:57.720180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083a3b4-304f-4483-b9a8-83c6ce2aa37d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:57.720180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083a3b4-304f-4483-b9a8-83c6ce2aa37d None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:57.720180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:15:57.721504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6df4-2f6e-4ea0-91d2-b7327ec53167 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:57.721916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6df4-2f6e-4ea0-91d2-b7327ec53167 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:57.722051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6df4-2f6e-4ea0-91d2-b7327ec53167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:57.789143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e81f6df4-2f6e-4ea0-91d2-b7327ec53167 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:15:57.790333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e81f6df4-2f6e-4ea0-91d2-b7327ec53167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:57.798474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:57.884092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:15:57.921589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083a3b4-304f-4483-b9a8-83c6ce2aa37d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:15:57.923187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5083a3b4-304f-4483-b9a8-83c6ce2aa37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:58.024486 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083a3b4-304f-4483-b9a8-83c6ce2aa37d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:15:58.077888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:15:58.096047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19115c9-ef65-49de-b32c-01d137624a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:15:58.096640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19115c9-ef65-49de-b32c-01d137624a03 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:15:58.096916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19115c9-ef65-49de-b32c-01d137624a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:15:58.097182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19115c9-ef65-49de-b32c-01d137624a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:15:58.099430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e19115c9-ef65-49de-b32c-01d137624a03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:15:58.206623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19115c9-ef65-49de-b32c-01d137624a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:15:58.208169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19115c9-ef65-49de-b32c-01d137624a03 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:15:58.226610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 733/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:00.284436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1ffcb-59af-4cea-afad-3b0b9517ad0e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:00.285007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1ffcb-59af-4cea-afad-3b0b9517ad0e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:00.285007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1ffcb-59af-4cea-afad-3b0b9517ad0e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:00.286983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:00.292260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe8eb8-4f3c-4717-b177-73270b089691 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:00.292670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe8eb8-4f3c-4717-b177-73270b089691 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:00.292802 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe8eb8-4f3c-4717-b177-73270b089691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:00.357694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6ebe8eb8-4f3c-4717-b177-73270b089691 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:00.358896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6ebe8eb8-4f3c-4717-b177-73270b089691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:00.397922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:00.455330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:00.493343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1ffcb-59af-4cea-afad-3b0b9517ad0e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:00.494880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-17d1ffcb-59af-4cea-afad-3b0b9517ad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:00.599095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1ffcb-59af-4cea-afad-3b0b9517ad0e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:00.636423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:00.672776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135315c-df64-41f6-a176-8aa3163c952c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:00.673375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135315c-df64-41f6-a176-8aa3163c952c service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:00.673652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135315c-df64-41f6-a176-8aa3163c952c service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:00.673922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135315c-df64-41f6-a176-8aa3163c952c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:00.675541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1135315c-df64-41f6-a176-8aa3163c952c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:00.785302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135315c-df64-41f6-a176-8aa3163c952c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:00.786760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135315c-df64-41f6-a176-8aa3163c952c service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:00.805699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 730/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:02.814164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555bdbd-c615-4f19-a9f5-9bb760348ea6 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:02.814662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555bdbd-c615-4f19-a9f5-9bb760348ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:02.814780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555bdbd-c615-4f19-a9f5-9bb760348ea6 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:02.816084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:02.822088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc564fcf-0869-4c74-990c-6259f39d5382 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:02.822527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc564fcf-0869-4c74-990c-6259f39d5382 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:02.822636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc564fcf-0869-4c74-990c-6259f39d5382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:02.887942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cc564fcf-0869-4c74-990c-6259f39d5382 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:02.889145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cc564fcf-0869-4c74-990c-6259f39d5382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:02.907417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:02.984132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:03.021854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555bdbd-c615-4f19-a9f5-9bb760348ea6 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:03.033000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e555bdbd-c615-4f19-a9f5-9bb760348ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:03.128190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555bdbd-c615-4f19-a9f5-9bb760348ea6 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:03.194601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:03.201147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfb926-1bce-4257-85bd-569e34b66705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:03.211662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfb926-1bce-4257-85bd-569e34b66705 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:03.211662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfb926-1bce-4257-85bd-569e34b66705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:03.211662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfb926-1bce-4257-85bd-569e34b66705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:03.211662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6fbfb926-1bce-4257-85bd-569e34b66705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:03.310930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfb926-1bce-4257-85bd-569e34b66705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:03.312322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfb926-1bce-4257-85bd-569e34b66705 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:03.331034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 736/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:05.356389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ba3ed-d06f-434a-aafd-fed91ea1f24e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:05.357034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ba3ed-d06f-434a-aafd-fed91ea1f24e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:05.357034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ba3ed-d06f-434a-aafd-fed91ea1f24e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:05.358299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:05.363640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546474f-e3ae-4cf5-a9d6-bcce116f2f87 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:05.364066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546474f-e3ae-4cf5-a9d6-bcce116f2f87 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:05.364189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546474f-e3ae-4cf5-a9d6-bcce116f2f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:05.432706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0546474f-e3ae-4cf5-a9d6-bcce116f2f87 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:05.433918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0546474f-e3ae-4cf5-a9d6-bcce116f2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:05.481641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:05.530190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:05.568252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ba3ed-d06f-434a-aafd-fed91ea1f24e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:05.569804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-548ba3ed-d06f-434a-aafd-fed91ea1f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:05.672634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ba3ed-d06f-434a-aafd-fed91ea1f24e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:05.725221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:05.751843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d38fd-a7cf-41bf-af04-92062c7143f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:05.752442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d38fd-a7cf-41bf-af04-92062c7143f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:05.752720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d38fd-a7cf-41bf-af04-92062c7143f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:05.752993 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d38fd-a7cf-41bf-af04-92062c7143f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:05.754584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2c6d38fd-a7cf-41bf-af04-92062c7143f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:05.863091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d38fd-a7cf-41bf-af04-92062c7143f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:05.864581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d38fd-a7cf-41bf-af04-92062c7143f3 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:05.883178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 733/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:16:15.528428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=22 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:15.880989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=66 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:25.417970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7d1be-a66b-486d-b9de-16374208241a None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:25.418559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7d1be-a66b-486d-b9de-16374208241a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:25.418559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7d1be-a66b-486d-b9de-16374208241a None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:25.419884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 738/1471] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:25.425158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b8a28-01fe-46e3-ba6a-5304c69bf0c0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:25.425605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b8a28-01fe-46e3-ba6a-5304c69bf0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:25.425724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b8a28-01fe-46e3-ba6a-5304c69bf0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:25.492598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8e7b8a28-01fe-46e3-ba6a-5304c69bf0c0 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:25.493801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8e7b8a28-01fe-46e3-ba6a-5304c69bf0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:25.588934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 734/1472] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:25.671540 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804097ed-8fd6-459f-ac57-ce8d004ea98d None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:25.671791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804097ed-8fd6-459f-ac57-ce8d004ea98d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:25.672067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804097ed-8fd6-459f-ac57-ce8d004ea98d None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:25.673425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 739/1473] 10.222.0.30 () {54 vars in 966 bytes} [Sun Aug 7 12:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:25.678209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cb73c-7e1f-4713-a867-2f184a440745 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:25.678499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cb73c-7e1f-4713-a867-2f184a440745 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:25.678778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cb73c-7e1f-4713-a867-2f184a440745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:25.754827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c90cb73c-7e1f-4713-a867-2f184a440745 None None] MFA Rules not processed for user `6b5ee9e4388d4bbabc7f1e9e3ca50b67`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:25.756095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c90cb73c-7e1f-4713-a867-2f184a440745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:25.777038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:25.833068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 735/1474] 10.222.0.30 () {58 vars in 1063 bytes} [Sun Aug 7 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:25.840617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804097ed-8fd6-459f-ac57-ce8d004ea98d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:25.842111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-804097ed-8fd6-459f-ac57-ce8d004ea98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:25.943634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804097ed-8fd6-459f-ac57-ce8d004ea98d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:26.015433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee5734-5aed-44ce-b28d-40e94afc86e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b5ee9e4388d4bbabc7f1e9e3ca50b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:26.016026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee5734-5aed-44ce-b28d-40e94afc86e5 service glance] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:26.016303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee5734-5aed-44ce-b28d-40e94afc86e5 service glance] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:26.016574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee5734-5aed-44ce-b28d-40e94afc86e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:26.018136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-89ee5734-5aed-44ce-b28d-40e94afc86e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:26.071167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:26.125483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee5734-5aed-44ce-b28d-40e94afc86e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:26.126946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee5734-5aed-44ce-b28d-40e94afc86e5 service glance] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:26.146251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 740/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:28.229644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6de4c-2cab-4d3d-8020-c37c9a831adf None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:28.230225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6de4c-2cab-4d3d-8020-c37c9a831adf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:28.230225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6de4c-2cab-4d3d-8020-c37c9a831adf None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:28.231599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 736/1476] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:28.237034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175853-81f7-433a-8dca-f8ddba5a64ed None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:28.237455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175853-81f7-433a-8dca-f8ddba5a64ed None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:28.237580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175853-81f7-433a-8dca-f8ddba5a64ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:28.302716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1b175853-81f7-433a-8dca-f8ddba5a64ed None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:28.303953 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1b175853-81f7-433a-8dca-f8ddba5a64ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:28.400284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 741/1477] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:28.423428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6de4c-2cab-4d3d-8020-c37c9a831adf None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:28.424948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dcc6de4c-2cab-4d3d-8020-c37c9a831adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:28.550097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6de4c-2cab-4d3d-8020-c37c9a831adf None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:28.598920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:28.644031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c5370-396e-4a11-b1d3-1aad9bc254f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:28.644637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5370-396e-4a11-b1d3-1aad9bc254f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:28.644947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5370-396e-4a11-b1d3-1aad9bc254f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:28.645222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5370-396e-4a11-b1d3-1aad9bc254f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:28.645871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c5370-396e-4a11-b1d3-1aad9bc254f4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:28.647624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-bf6c5370-396e-4a11-b1d3-1aad9bc254f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:16:28.647624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Aug 07 12:16:28.647624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: Aug 07 12:16:28.648757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c5370-396e-4a11-b1d3-1aad9bc254f4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:28.667783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 737/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Sun Aug 7 12:16:28 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:16:30.668388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb9d79-d4d4-4b76-a5a3-a881d067ce15 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:30.669007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb9d79-d4d4-4b76-a5a3-a881d067ce15 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:30.669007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb9d79-d4d4-4b76-a5a3-a881d067ce15 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:30.670258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 742/1479] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:30.675375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9383acf-1167-4e85-9c6b-0476606c2e9b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:30.675784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9383acf-1167-4e85-9c6b-0476606c2e9b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:30.675927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9383acf-1167-4e85-9c6b-0476606c2e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:30.741987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c9383acf-1167-4e85-9c6b-0476606c2e9b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:30.743227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c9383acf-1167-4e85-9c6b-0476606c2e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:30.839520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 738/1480] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:30.883173 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb9d79-d4d4-4b76-a5a3-a881d067ce15 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:30.884763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-49cb9d79-d4d4-4b76-a5a3-a881d067ce15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:30.894615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:31.028869 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb9d79-d4d4-4b76-a5a3-a881d067ce15 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:31.107905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:31.140407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71943ae-7d13-4efc-a7f3-335a05b3e79a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:31.141282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71943ae-7d13-4efc-a7f3-335a05b3e79a service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:31.141724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71943ae-7d13-4efc-a7f3-335a05b3e79a service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:31.142041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71943ae-7d13-4efc-a7f3-335a05b3e79a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:31.143940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e71943ae-7d13-4efc-a7f3-335a05b3e79a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:31.231306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71943ae-7d13-4efc-a7f3-335a05b3e79a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:31.232922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71943ae-7d13-4efc-a7f3-335a05b3e79a service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:31.250260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 743/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:33.427193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075391ab-2a4f-4d57-b1af-6cd5e71d7e27 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:33.427737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075391ab-2a4f-4d57-b1af-6cd5e71d7e27 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:33.427737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075391ab-2a4f-4d57-b1af-6cd5e71d7e27 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:33.429102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 739/1482] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:33.434297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72130188-2fb1-4edd-aa44-53fac59d78df None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:33.434748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72130188-2fb1-4edd-aa44-53fac59d78df None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:33.434877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72130188-2fb1-4edd-aa44-53fac59d78df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:33.500647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-72130188-2fb1-4edd-aa44-53fac59d78df None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:33.501870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-72130188-2fb1-4edd-aa44-53fac59d78df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:33.547878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:33.598670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 744/1483] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:33.635278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075391ab-2a4f-4d57-b1af-6cd5e71d7e27 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:33.636812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-075391ab-2a4f-4d57-b1af-6cd5e71d7e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:33.660013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:33.741463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075391ab-2a4f-4d57-b1af-6cd5e71d7e27 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:33.813092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b49fdb-ab3d-421f-bdc8-1643d9840d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:33.813687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49fdb-ab3d-421f-bdc8-1643d9840d3a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:33.813962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49fdb-ab3d-421f-bdc8-1643d9840d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:33.814280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49fdb-ab3d-421f-bdc8-1643d9840d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:33.815882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-26b49fdb-ab3d-421f-bdc8-1643d9840d3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:33.891368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:33.927091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b49fdb-ab3d-421f-bdc8-1643d9840d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:33.928550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b49fdb-ab3d-421f-bdc8-1643d9840d3a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:33.945979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 740/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:36.011478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cece9d-0447-4ecf-bae4-84318f29aea8 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:36.012002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cece9d-0447-4ecf-bae4-84318f29aea8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:36.012002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cece9d-0447-4ecf-bae4-84318f29aea8 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:36.013276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 745/1485] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:36.018481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f457a-6292-4777-a029-a5cbc6a7cfb1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:36.018893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f457a-6292-4777-a029-a5cbc6a7cfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:36.019058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f457a-6292-4777-a029-a5cbc6a7cfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:36.086949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4a6f457a-6292-4777-a029-a5cbc6a7cfb1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:36.088200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4a6f457a-6292-4777-a029-a5cbc6a7cfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:36.185196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 741/1486] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:36.223671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cece9d-0447-4ecf-bae4-84318f29aea8 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:36.225215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-26cece9d-0447-4ecf-bae4-84318f29aea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:36.326482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cece9d-0447-4ecf-bae4-84318f29aea8 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:36.398028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:36.399932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a3dd1-a91c-40ea-bc81-43c76fa347da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:36.400539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a3dd1-a91c-40ea-bc81-43c76fa347da service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:36.400958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a3dd1-a91c-40ea-bc81-43c76fa347da service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:36.401235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a3dd1-a91c-40ea-bc81-43c76fa347da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:36.402866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1b0a3dd1-a91c-40ea-bc81-43c76fa347da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:36.513048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a3dd1-a91c-40ea-bc81-43c76fa347da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:36.514473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a3dd1-a91c-40ea-bc81-43c76fa347da service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:36.532559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 746/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:38.621419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cc6ac-c8f4-415b-89d2-7c8e6332fa02 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:38.622003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cc6ac-c8f4-415b-89d2-7c8e6332fa02 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:38.622003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cc6ac-c8f4-415b-89d2-7c8e6332fa02 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:38.623354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 742/1488] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:38.628705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e0ad7-3d08-474f-9016-cc014fc6b2e6 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:38.629733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e0ad7-3d08-474f-9016-cc014fc6b2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:38.629833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e0ad7-3d08-474f-9016-cc014fc6b2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:38.698891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-247e0ad7-3d08-474f-9016-cc014fc6b2e6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:38.700134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-247e0ad7-3d08-474f-9016-cc014fc6b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:38.752777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:38.797035 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 747/1489] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:16:38.837273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cc6ac-c8f4-415b-89d2-7c8e6332fa02 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:38.838818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-888cc6ac-c8f4-415b-89d2-7c8e6332fa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:38.874275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:38.944407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cc6ac-c8f4-415b-89d2-7c8e6332fa02 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:39.019706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2037-ee76-4d69-acc9-e36d86932bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:39.020318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2037-ee76-4d69-acc9-e36d86932bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:39.020598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2037-ee76-4d69-acc9-e36d86932bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:39.020876 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2037-ee76-4d69-acc9-e36d86932bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:39.022486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0e7a2037-ee76-4d69-acc9-e36d86932bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:39.103296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:39.136983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a2037-ee76-4d69-acc9-e36d86932bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:39.138332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a2037-ee76-4d69-acc9-e36d86932bf7 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:39.156606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 743/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:41.530242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ba66-c3fb-4fed-8763-3a238aadedd8 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:41.530820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ba66-c3fb-4fed-8763-3a238aadedd8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:41.530820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ba66-c3fb-4fed-8763-3a238aadedd8 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:41.532144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 748/1491] 10.222.0.30 () {54 vars in 927 bytes} [Sun Aug 7 12:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:16:41.538342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d2832-44a3-46ec-b587-5b868400a667 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:41.538782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d2832-44a3-46ec-b587-5b868400a667 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:41.538894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d2832-44a3-46ec-b587-5b868400a667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:41.610842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-bb7d2832-44a3-46ec-b587-5b868400a667 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:16:41.612373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bb7d2832-44a3-46ec-b587-5b868400a667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:41.717874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 744/1492] 10.222.0.30 () {58 vars in 1024 bytes} [Sun Aug 7 12:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:16:41.763167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307ba66-c3fb-4fed-8763-3a238aadedd8 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:16:41.764706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a307ba66-c3fb-4fed-8763-3a238aadedd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:41.876529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307ba66-c3fb-4fed-8763-3a238aadedd8 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:16:41.948945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eef8e5-8447-463d-a6f0-bc8cf7b3ebdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:16:41.949544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef8e5-8447-463d-a6f0-bc8cf7b3ebdd service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:16:41.949824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef8e5-8447-463d-a6f0-bc8cf7b3ebdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:16:41.950094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef8e5-8447-463d-a6f0-bc8cf7b3ebdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:16:41.951717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-90eef8e5-8447-463d-a6f0-bc8cf7b3ebdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:16:41.961370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:42.061617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eef8e5-8447-463d-a6f0-bc8cf7b3ebdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:16:42.063428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eef8e5-8447-463d-a6f0-bc8cf7b3ebdd service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:16:42.081373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 749/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:16:51.715933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=82 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:16:52.079047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=50 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:20.008338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562ac00-65b6-474c-b36a-599cf48bce49 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:20.008947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562ac00-65b6-474c-b36a-599cf48bce49 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:20.008947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562ac00-65b6-474c-b36a-599cf48bce49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:20.083122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e562ac00-65b6-474c-b36a-599cf48bce49 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:18:20.084316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e562ac00-65b6-474c-b36a-599cf48bce49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:20.181429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 745/1494] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:18:20.189505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b15787-2735-486f-8b42-340ab68c0264 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:20.189997 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b15787-2735-486f-8b42-340ab68c0264 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:20.190101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b15787-2735-486f-8b42-340ab68c0264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:20.265151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e6b15787-2735-486f-8b42-340ab68c0264 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:18:20.266314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e6b15787-2735-486f-8b42-340ab68c0264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:20.362175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 750/1495] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:18:20.367099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562ac00-65b6-474c-b36a-599cf48bce49 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:20.368330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e562ac00-65b6-474c-b36a-599cf48bce49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:20.379905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562ac00-65b6-474c-b36a-599cf48bce49 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:20.440237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:20.474395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:20.474994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:20.475298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:20.475568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:20.476205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:18:20.476503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:20.477521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:18:20.477521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Aug 07 12:18:20.477521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: Aug 07 12:18:20.478600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:20.490022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 746/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:18:20.498287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b15787-2735-486f-8b42-340ab68c0264 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:20.507746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b15787-2735-486f-8b42-340ab68c0264 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:20.569933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:20.604927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:20.605518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:20.605793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:20.606070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:20.606675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:20.608422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING py.warnings [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:18:20.608422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: warnings.warn(msg) Aug 07 12:18:20.608422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: Aug 07 12:18:20.609598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:20.714443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 751/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:18:20 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:18:20.718638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:20.720314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:20.755451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97c7af-199c-49fb-ad24-f464e5cc9dff admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:20.849322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:20.849912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:20.850188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:20.850454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:20.851092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:20.852093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:18:20.852093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Aug 07 12:18:20.852093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: Aug 07 12:18:20.853281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:20.936207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 747/1498] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:18:20 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:18:20.940442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:20.944606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:21.011387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d48ac-1584-4916-96eb-4f3adae7edc9 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:21.092057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:21.102039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:21.102637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:21.102909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:21.103202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:21.103843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:21.104865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING py.warnings [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:18:21.104865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: warnings.warn(msg) Aug 07 12:18:21.104865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: Aug 07 12:18:21.105742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:21.119949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 752/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:18:21.123959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:21.125485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:21.137943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a681e-3e0d-4606-b8a3-70ce940094cb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:21.163185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:21.234409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addae116-9fad-42b5-8453-5ec7820ecd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:21.235092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addae116-9fad-42b5-8453-5ec7820ecd5e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:21.235308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addae116-9fad-42b5-8453-5ec7820ecd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:21.235575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addae116-9fad-42b5-8453-5ec7820ecd5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:21.236204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addae116-9fad-42b5-8453-5ec7820ecd5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:21.237204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-addae116-9fad-42b5-8453-5ec7820ecd5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:18:21.237204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Aug 07 12:18:21.237204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: Aug 07 12:18:21.238776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addae116-9fad-42b5-8453-5ec7820ecd5e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:21.250940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 748/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:18:21.256161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:21.257682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:21.290478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379062a-1aa7-4ad0-9e5c-cc18b28e40b4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:21.383194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33ba8d-43dd-464f-8a6e-d3c85b7ae64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:21.383843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ba8d-43dd-464f-8a6e-d3c85b7ae64f admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:21.384118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ba8d-43dd-464f-8a6e-d3c85b7ae64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:21.384387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ba8d-43dd-464f-8a6e-d3c85b7ae64f admin admin] PATH_INFO: `/v3/projects/aa5585bf455946bf8846dd58654e3c26/users/01a7df9873104675aa87e4955f9dc53b/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:21.405379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33ba8d-43dd-464f-8a6e-d3c85b7ae64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5585bf455946bf8846dd58654e3c26, user_id=01a7df9873104675aa87e4955f9dc53b, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:21.406482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING py.warnings [None req-8b33ba8d-43dd-464f-8a6e-d3c85b7ae64f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:18:21.406482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: warnings.warn(msg) Aug 07 12:18:21.406482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: Aug 07 12:18:21.408444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33ba8d-43dd-464f-8a6e-d3c85b7ae64f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:21.501361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 753/1501] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:18:21 2022] PUT /identity/v3/projects/aa5585bf455946bf8846dd58654e3c26/users/01a7df9873104675aa87e4955f9dc53b/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:18:21.513996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deed9aa-cfe6-49df-a79b-e128ceb75053 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:21.514308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deed9aa-cfe6-49df-a79b-e128ceb75053 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:21.514614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deed9aa-cfe6-49df-a79b-e128ceb75053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:21.572899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4deed9aa-cfe6-49df-a79b-e128ceb75053 None None] MFA Rules not processed for user `01a7df9873104675aa87e4955f9dc53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:18:21.574108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4deed9aa-cfe6-49df-a79b-e128ceb75053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:21.649126 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 749/1502] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:18:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:18:21.662651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240a34d-2d4f-485f-8e28-16ef00d95e4d None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:21.662932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240a34d-2d4f-485f-8e28-16ef00d95e4d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:21.663236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240a34d-2d4f-485f-8e28-16ef00d95e4d None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:21.664608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 754/1503] 10.222.0.30 () {54 vars in 964 bytes} [Sun Aug 7 12:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:18:21.669207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c845-71c0-4c44-bce2-fb76d66df5ba None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:21.669498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c845-71c0-4c44-bce2-fb76d66df5ba None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:21.669778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c845-71c0-4c44-bce2-fb76d66df5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:21.701323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:21.747153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ea42c845-71c0-4c44-bce2-fb76d66df5ba None None] MFA Rules not processed for user `e0792940b93b4c1998be971f6143a514`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:18:21.748364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ea42c845-71c0-4c44-bce2-fb76d66df5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:21.823094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 750/1504] 10.222.0.30 () {58 vars in 1061 bytes} [Sun Aug 7 12:18:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:18:21.830749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240a34d-2d4f-485f-8e28-16ef00d95e4d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:21.832787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0240a34d-2d4f-485f-8e28-16ef00d95e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:21.881376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:21.933526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240a34d-2d4f-485f-8e28-16ef00d95e4d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:22.005830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:22.006424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:22.006699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:22.006970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:22.008557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:22.104772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:22.106245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:22.109861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:22.124277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 755/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:18:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:18:22.169880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7a84-6658-4a0c-8850-341aa24b29ac None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:22.170332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7a84-6658-4a0c-8850-341aa24b29ac None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:22.170463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7a84-6658-4a0c-8850-341aa24b29ac None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:22.171725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 751/1506] 10.222.0.30 () {52 vars in 830 bytes} [Sun Aug 7 12:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:18:22.175294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:22.176577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:22.274001 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c8f3e-824d-45d3-ac1c-47a34bc4df00 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:22.364839 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:22.365457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:22.365732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:22.366005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] PATH_INFO: `/v3/users/01a7df9873104675aa87e4955f9dc53b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:22.377944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a7df9873104675aa87e4955f9dc53b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:22.379315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING py.warnings [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:18:22.379315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: warnings.warn(msg) Aug 07 12:18:22.379315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: Aug 07 12:18:22.380518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:22.390141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:22.543912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a7df9873104675aa87e4955f9dc53b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:22.567098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a7df9873104675aa87e4955f9dc53b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:22.581187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a7df9873104675aa87e4955f9dc53b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:22.582312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a7df9873104675aa87e4955f9dc53b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:22.582746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a7df9873104675aa87e4955f9dc53b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:22.642569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 756/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:18:22 2022] DELETE /identity/v3/users/01a7df9873104675aa87e4955f9dc53b => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:18:22.649740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5bbd-d344-4e16-9621-3b7dc2e5b002 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:22.650030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5bbd-d344-4e16-9621-3b7dc2e5b002 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:22.650305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5bbd-d344-4e16-9621-3b7dc2e5b002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:22.707013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b03b5bbd-d344-4e16-9621-3b7dc2e5b002 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:18:22.708381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b03b5bbd-d344-4e16-9621-3b7dc2e5b002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:22.782974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:22.796134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 752/1508] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:18:22.805952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:22.807290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:22.907431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f4fd3-ab14-45ce-8a37-fb4146206d1a admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:22.961375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:22.981007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88829a-34e8-4b74-bd51-f0a7a3d3b098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:22.981613 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88829a-34e8-4b74-bd51-f0a7a3d3b098 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:22.981918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88829a-34e8-4b74-bd51-f0a7a3d3b098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:22.982191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88829a-34e8-4b74-bd51-f0a7a3d3b098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:22.983817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8d88829a-34e8-4b74-bd51-f0a7a3d3b098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:23.092117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88829a-34e8-4b74-bd51-f0a7a3d3b098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:23.093458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88829a-34e8-4b74-bd51-f0a7a3d3b098 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:23.112314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 757/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:18:23.531291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5bbd-d344-4e16-9621-3b7dc2e5b002 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:18:23.536791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b03b5bbd-d344-4e16-9621-3b7dc2e5b002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:18:23.544788 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5bbd-d344-4e16-9621-3b7dc2e5b002 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:18:23.641537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:18:23.642162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:18:23.642439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:18:23.642706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] PATH_INFO: `/v3/projects/aa5585bf455946bf8846dd58654e3c26` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:18:23.653551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5585bf455946bf8846dd58654e3c26)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:18:23.654561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:18:23.654561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Aug 07 12:18:23.654561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: Aug 07 12:18:23.655780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:18:23.763431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5585bf455946bf8846dd58654e3c26', 'request_id': 'req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:23.826192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5585bf455946bf8846dd58654e3c26', 'request_id': 'req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:23.833648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5585bf455946bf8846dd58654e3c26', 'request_id': 'req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:23.864206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] The token cache is being invalidate because project aa5585bf455946bf8846dd58654e3c26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:18:23.864497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:23.865023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2b7be5ed-1696-47bb-9099-0f6df3c2b17e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:18:23.866319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 753/1510] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:18:23 2022] DELETE /identity/v3/projects/aa5585bf455946bf8846dd58654e3c26 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:18:33.113949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=62 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:18:33.862120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:19:49.841605 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2f82-5e7b-48d4-b8c7-7239092f988d None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:19:49.842160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2f82-5e7b-48d4-b8c7-7239092f988d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:19:49.842160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2f82-5e7b-48d4-b8c7-7239092f988d None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:19:49.843550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:19:49.848475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68267c-f4d5-40cb-ae58-e2438a7a89bf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:19:49.848905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68267c-f4d5-40cb-ae58-e2438a7a89bf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:19:49.849092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68267c-f4d5-40cb-ae58-e2438a7a89bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:19:49.931354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fa68267c-f4d5-40cb-ae58-e2438a7a89bf None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:19:49.932560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fa68267c-f4d5-40cb-ae58-e2438a7a89bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:19:50.034254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:19:50.185795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2f82-5e7b-48d4-b8c7-7239092f988d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:19:50.188108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f59e2f82-5e7b-48d4-b8c7-7239092f988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:19:50.307368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2f82-5e7b-48d4-b8c7-7239092f988d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:19:50.312875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faf804-e21c-4680-b770-73026d2711b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:19:50.313473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faf804-e21c-4680-b770-73026d2711b0 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:19:50.313755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faf804-e21c-4680-b770-73026d2711b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:19:50.314029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faf804-e21c-4680-b770-73026d2711b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:19:50.322519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faf804-e21c-4680-b770-73026d2711b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:19:50.324133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faf804-e21c-4680-b770-73026d2711b0 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:19:50.333376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 759/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:19:51.177866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164029-84d4-4783-944b-fcadc1f6e58d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:19:51.178396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164029-84d4-4783-944b-fcadc1f6e58d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:19:51.178396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce164029-84d4-4783-944b-fcadc1f6e58d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:19:51.179713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:19:51.184640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2924-ebe3-40e5-93ac-a91561a70592 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:19:51.185042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2924-ebe3-40e5-93ac-a91561a70592 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:19:51.185208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2924-ebe3-40e5-93ac-a91561a70592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:19:51.204378 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d0dc2924-ebe3-40e5-93ac-a91561a70592 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:19:51.205581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d0dc2924-ebe3-40e5-93ac-a91561a70592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:19:51.214027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:19:51.357517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4220-7e36-46ff-a5b3-a66fe18c82aa None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:19:51.357797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4220-7e36-46ff-a5b3-a66fe18c82aa None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:19:51.358283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4220-7e36-46ff-a5b3-a66fe18c82aa None None] PATH_INFO: `/v3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:19:51.359590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 756/1516] 10.222.0.18 () {54 vars in 925 bytes} [Sun Aug 7 12:19:51 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:19:51.366704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156e030-3f14-4998-9365-3edb0d300ca3 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:19:51.366994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156e030-3f14-4998-9365-3edb0d300ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:19:51.367306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156e030-3f14-4998-9365-3edb0d300ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:19:51.367409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f4220-7e36-46ff-a5b3-a66fe18c82aa None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:19:51.382254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f4220-7e36-46ff-a5b3-a66fe18c82aa None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:19:51.387837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98d74f-1437-4a5e-8392-7546d54ca433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:19:51.388429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98d74f-1437-4a5e-8392-7546d54ca433 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:19:51.388708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98d74f-1437-4a5e-8392-7546d54ca433 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:19:51.388987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98d74f-1437-4a5e-8392-7546d54ca433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:19:51.398036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98d74f-1437-4a5e-8392-7546d54ca433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:19:51.399662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98d74f-1437-4a5e-8392-7546d54ca433 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:19:51.409166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 757/1517] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:19:51.432500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7156e030-3f14-4998-9365-3edb0d300ca3 None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:19:51.433641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7156e030-3f14-4998-9365-3edb0d300ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:19:51.492531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:19:51.513350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 761/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Sun Aug 7 12:19:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:20:00.516260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315907-54a4-420e-b47f-a45af8528cbb None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:00.516826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315907-54a4-420e-b47f-a45af8528cbb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:00.516826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315907-54a4-420e-b47f-a45af8528cbb None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:00.518133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 758/1519] 10.222.0.30 () {54 vars in 961 bytes} [Sun Aug 7 12:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:20:00.524008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86eb1e-8376-461b-a7b3-b5762a87f975 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:00.524452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86eb1e-8376-461b-a7b3-b5762a87f975 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:00.524563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86eb1e-8376-461b-a7b3-b5762a87f975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:00.543266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4a86eb1e-8376-461b-a7b3-b5762a87f975 None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:20:00.544451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4a86eb1e-8376-461b-a7b3-b5762a87f975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:00.552891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 762/1520] 10.222.0.30 () {58 vars in 1058 bytes} [Sun Aug 7 12:20:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:20:00.560262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0315907-54a4-420e-b47f-a45af8528cbb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:20:00.573876 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0315907-54a4-420e-b47f-a45af8528cbb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:20:00.578744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f42464-4ed7-42d5-b902-8eb475d4d182 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:20:00.579471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f42464-4ed7-42d5-b902-8eb475d4d182 service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:00.579757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f42464-4ed7-42d5-b902-8eb475d4d182 service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:00.580083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f42464-4ed7-42d5-b902-8eb475d4d182 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:00.589170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f42464-4ed7-42d5-b902-8eb475d4d182 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:20:00.590723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f42464-4ed7-42d5-b902-8eb475d4d182 service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:20:00.600276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 759/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:20:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:20:02.629316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca8dd8-9d85-4538-a653-0309204dbbc6 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:02.629906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca8dd8-9d85-4538-a653-0309204dbbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:02.629906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca8dd8-9d85-4538-a653-0309204dbbc6 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:02.632368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:20:02.640745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62407cc-2f41-4387-9c1b-9eda32aefcd1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:02.641198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62407cc-2f41-4387-9c1b-9eda32aefcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:02.641319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62407cc-2f41-4387-9c1b-9eda32aefcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:02.660673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d62407cc-2f41-4387-9c1b-9eda32aefcd1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:20:02.661924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d62407cc-2f41-4387-9c1b-9eda32aefcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:02.669623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:20:02.823611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca8dd8-9d85-4538-a653-0309204dbbc6 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:20:02.838670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca8dd8-9d85-4538-a653-0309204dbbc6 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:20:02.844547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e753a-dc38-4011-8aff-9563df6a504a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:20:02.844681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e753a-dc38-4011-8aff-9563df6a504a service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:02.844902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e753a-dc38-4011-8aff-9563df6a504a service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:02.845171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e753a-dc38-4011-8aff-9563df6a504a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:02.853029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e753a-dc38-4011-8aff-9563df6a504a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:20:02.854598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e753a-dc38-4011-8aff-9563df6a504a service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:20:02.863729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 764/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:20:03.681920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4d38a-b4eb-4a91-85b5-b0a682209748 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:03.682470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4d38a-b4eb-4a91-85b5-b0a682209748 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:03.682587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4d38a-b4eb-4a91-85b5-b0a682209748 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:03.684005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 12:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:20:03.689244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5431a73-8bfd-4689-8401-aba9aec3b5e3 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:03.689680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5431a73-8bfd-4689-8401-aba9aec3b5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:03.689790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5431a73-8bfd-4689-8401-aba9aec3b5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:03.709527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e5431a73-8bfd-4689-8401-aba9aec3b5e3 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:20:03.710720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e5431a73-8bfd-4689-8401-aba9aec3b5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:03.718545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 12:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:20:03.871339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4d38a-b4eb-4a91-85b5-b0a682209748 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:20:03.885932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4d38a-b4eb-4a91-85b5-b0a682209748 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:20:03.891311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c9d76-adf7-4d8e-9402-bf26e86fd2a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:20:03.891897 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9d76-adf7-4d8e-9402-bf26e86fd2a6 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:03.892178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9d76-adf7-4d8e-9402-bf26e86fd2a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:03.892450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9d76-adf7-4d8e-9402-bf26e86fd2a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:03.900831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c9d76-adf7-4d8e-9402-bf26e86fd2a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:20:03.902385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c9d76-adf7-4d8e-9402-bf26e86fd2a6 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:20:03.912421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 762/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:20:07.713644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026924e-43b5-4021-989b-88ebd1484474 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:07.714254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026924e-43b5-4021-989b-88ebd1484474 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:07.714254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026924e-43b5-4021-989b-88ebd1484474 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:07.715641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:20:07.720847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59c18-66c6-48ed-b9d3-d889a047e526 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:07.721274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59c18-66c6-48ed-b9d3-d889a047e526 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:07.721378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59c18-66c6-48ed-b9d3-d889a047e526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:07.741688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0cf59c18-66c6-48ed-b9d3-d889a047e526 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:20:07.742869 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0cf59c18-66c6-48ed-b9d3-d889a047e526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:07.750505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:20:07.823727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026924e-43b5-4021-989b-88ebd1484474 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:20:07.825455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0026924e-43b5-4021-989b-88ebd1484474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:07.901585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026924e-43b5-4021-989b-88ebd1484474 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:20:07.905111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f64ae-c610-4d95-aeee-02e3c9d516d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:20:07.905704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f64ae-c610-4d95-aeee-02e3c9d516d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:07.905979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f64ae-c610-4d95-aeee-02e3c9d516d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:07.906252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f64ae-c610-4d95-aeee-02e3c9d516d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:07.915390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f64ae-c610-4d95-aeee-02e3c9d516d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:20:07.916810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f64ae-c610-4d95-aeee-02e3c9d516d6 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:20:07.925830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 767/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:20:09.906843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d8572-b21b-43e3-a6e5-237b47bcdf33 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:09.907448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d8572-b21b-43e3-a6e5-237b47bcdf33 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:09.907448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d8572-b21b-43e3-a6e5-237b47bcdf33 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:09.908771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:20:09.914254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143deab-d78d-4c71-a2d9-184012b517fa None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:09.914685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143deab-d78d-4c71-a2d9-184012b517fa None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:09.914796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143deab-d78d-4c71-a2d9-184012b517fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:09.938197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7143deab-d78d-4c71-a2d9-184012b517fa None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:20:09.939435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7143deab-d78d-4c71-a2d9-184012b517fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:09.947837 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:20:09.983368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d8572-b21b-43e3-a6e5-237b47bcdf33 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:20:09.994448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d8572-b21b-43e3-a6e5-237b47bcdf33 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:20:09.997856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fc446-b8fd-4ca3-8dcd-2434e3059eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:20:09.998440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc446-b8fd-4ca3-8dcd-2434e3059eca service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:09.998717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc446-b8fd-4ca3-8dcd-2434e3059eca service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:09.998985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc446-b8fd-4ca3-8dcd-2434e3059eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:10.007986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fc446-b8fd-4ca3-8dcd-2434e3059eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:20:10.009438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fc446-b8fd-4ca3-8dcd-2434e3059eca service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:20:10.018903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 765/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:20:13.580798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039134-48a8-4aeb-9ae5-75e02462d976 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:13.581392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039134-48a8-4aeb-9ae5-75e02462d976 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:13.581392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039134-48a8-4aeb-9ae5-75e02462d976 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:13.582696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:20:13.588301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510d790-bbbe-4591-bb7d-ce2ed5b19597 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:13.588736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510d790-bbbe-4591-bb7d-ce2ed5b19597 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:13.588904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510d790-bbbe-4591-bb7d-ce2ed5b19597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:13.596412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:20:13.608781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6510d790-bbbe-4591-bb7d-ce2ed5b19597 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:20:13.609981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6510d790-bbbe-4591-bb7d-ce2ed5b19597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:13.617990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:20:13.923139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039134-48a8-4aeb-9ae5-75e02462d976 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:20:13.924876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e5039134-48a8-4aeb-9ae5-75e02462d976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:14.001560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039134-48a8-4aeb-9ae5-75e02462d976 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:20:14.005647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85f36a-1d52-4a88-a1b0-e8a5625e1279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:20:14.006506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f36a-1d52-4a88-a1b0-e8a5625e1279 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:14.006785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f36a-1d52-4a88-a1b0-e8a5625e1279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:14.007088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f36a-1d52-4a88-a1b0-e8a5625e1279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:14.016012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f36a-1d52-4a88-a1b0-e8a5625e1279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:20:14.017476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f36a-1d52-4a88-a1b0-e8a5625e1279 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:20:14.026757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 770/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:20:16.030418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23384dc8-c134-41a7-a30a-edd9a45b4453 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:16.030989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23384dc8-c134-41a7-a30a-edd9a45b4453 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:16.030989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23384dc8-c134-41a7-a30a-edd9a45b4453 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:16.032397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 12:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:20:16.037595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6148cd-6587-4aba-97ee-911c2b0b7891 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:16.037997 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6148cd-6587-4aba-97ee-911c2b0b7891 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:16.038142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6148cd-6587-4aba-97ee-911c2b0b7891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:16.057293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7c6148cd-6587-4aba-97ee-911c2b0b7891 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:20:16.061470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7c6148cd-6587-4aba-97ee-911c2b0b7891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:16.066444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 12:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:20:16.367120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23384dc8-c134-41a7-a30a-edd9a45b4453 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:20:16.368799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-23384dc8-c134-41a7-a30a-edd9a45b4453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:20:16.382555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23384dc8-c134-41a7-a30a-edd9a45b4453 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:20:16.386028 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8124c9-e885-4c69-ba3a-345dd5ae45ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:20:16.386614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8124c9-e885-4c69-ba3a-345dd5ae45ef service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:20:16.386891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8124c9-e885-4c69-ba3a-345dd5ae45ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:20:16.387197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8124c9-e885-4c69-ba3a-345dd5ae45ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:20:16.395925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8124c9-e885-4c69-ba3a-345dd5ae45ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:20:16.397334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8124c9-e885-4c69-ba3a-345dd5ae45ef service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:20:16.406517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 768/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:20:26.052608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:20:26.402413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=8 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:39.447939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce22bd-4ede-4036-94d2-eff3a0b9c3ff None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.451714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce22bd-4ede-4036-94d2-eff3a0b9c3ff None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.452428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce22bd-4ede-4036-94d2-eff3a0b9c3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.483647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1fce22bd-4ede-4036-94d2-eff3a0b9c3ff None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:39.485269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1fce22bd-4ede-4036-94d2-eff3a0b9c3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:39.498579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 772/1540] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:21:39.508570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3b66b-5501-4239-942c-1caea025d3ea None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.509739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3b66b-5501-4239-942c-1caea025d3ea None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.509739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3b66b-5501-4239-942c-1caea025d3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.512758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09906-e3aa-416f-9024-31776b2eba89 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.513465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09906-e3aa-416f-9024-31776b2eba89 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.513955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09906-e3aa-416f-9024-31776b2eba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.538770 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-42f09906-e3aa-416f-9024-31776b2eba89 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:39.540554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-42f09906-e3aa-416f-9024-31776b2eba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:39.548107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c9e3b66b-5501-4239-942c-1caea025d3ea None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:39.549322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c9e3b66b-5501-4239-942c-1caea025d3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:39.556686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 773/1541] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:21:39.565493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 769/1542] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:21:39.570431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09906-e3aa-416f-9024-31776b2eba89 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:39.570969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116499-ccf4-4c88-9092-37fdbd1d9108 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.571292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116499-ccf4-4c88-9092-37fdbd1d9108 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.571575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116499-ccf4-4c88-9092-37fdbd1d9108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.585273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09906-e3aa-416f-9024-31776b2eba89 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:39.594958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:39.595922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.596467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.596948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.597803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:39.598312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:39.599689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING py.warnings [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:21:39.599689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: warnings.warn(msg) Aug 07 12:21:39.599689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: Aug 07 12:21:39.601162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:39.604436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-37116499-ccf4-4c88-9092-37fdbd1d9108 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:39.605684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-37116499-ccf4-4c88-9092-37fdbd1d9108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:39.621599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 774/1543] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:39.624937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 770/1544] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:21:39.629742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:39.636784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37116499-ccf4-4c88-9092-37fdbd1d9108 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:39.644346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a8cc-f041-4b46-9602-c34ddc17e569 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:39.655217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37116499-ccf4-4c88-9092-37fdbd1d9108 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:39.656384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:39.657414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.657901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.658374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.659350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:39.660026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:39.662104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:39.669944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d391f8-1178-4073-a6fe-0736d88e0d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:39.670590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d391f8-1178-4073-a6fe-0736d88e0d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.670872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d391f8-1178-4073-a6fe-0736d88e0d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.671178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d391f8-1178-4073-a6fe-0736d88e0d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.671840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d391f8-1178-4073-a6fe-0736d88e0d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:39.672885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-f1d391f8-1178-4073-a6fe-0736d88e0d19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:21:39.672885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Aug 07 12:21:39.672885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: Aug 07 12:21:39.674118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d391f8-1178-4073-a6fe-0736d88e0d19 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:39.684550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 775/1545] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:39.718956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:39.737434 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f423c3-23d8-4def-afa0-1374c2c9a00d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:39.748242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38388d5b-0166-4893-8c1d-2e9099e4a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:39.749385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38388d5b-0166-4893-8c1d-2e9099e4a6da admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.749880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38388d5b-0166-4893-8c1d-2e9099e4a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.750399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38388d5b-0166-4893-8c1d-2e9099e4a6da admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.751255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38388d5b-0166-4893-8c1d-2e9099e4a6da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:39.755836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38388d5b-0166-4893-8c1d-2e9099e4a6da admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:39.789727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 771/1546] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:21:39.800208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625d816-b30d-4e54-9451-d6ba7fec1bc1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.800208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625d816-b30d-4e54-9451-d6ba7fec1bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.800208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625d816-b30d-4e54-9451-d6ba7fec1bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.822170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4625d816-b30d-4e54-9451-d6ba7fec1bc1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:39.824118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4625d816-b30d-4e54-9451-d6ba7fec1bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:39.881297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 776/1547] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:39.882604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38388d5b-0166-4893-8c1d-2e9099e4a6da admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:39.927382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 772/1548] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:39.932478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926b4f-2081-47b2-937b-26bb8d27f5c5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.933103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926b4f-2081-47b2-937b-26bb8d27f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.933614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926b4f-2081-47b2-937b-26bb8d27f5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.957918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-62926b4f-2081-47b2-937b-26bb8d27f5c5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:39.959787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-62926b4f-2081-47b2-937b-26bb8d27f5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:39.980877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38388d5b-0166-4893-8c1d-2e9099e4a6da admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:39.985344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d0b00-a4d4-4c4e-abbe-52ec2aa70c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:39.985945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0b00-a4d4-4c4e-abbe-52ec2aa70c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:39.986225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0b00-a4d4-4c4e-abbe-52ec2aa70c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:39.986816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0b00-a4d4-4c4e-abbe-52ec2aa70c16 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:39.987143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d0b00-a4d4-4c4e-abbe-52ec2aa70c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:39.988407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING py.warnings [None req-bd3d0b00-a4d4-4c4e-abbe-52ec2aa70c16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:21:39.988407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: warnings.warn(msg) Aug 07 12:21:39.988407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: Aug 07 12:21:39.989602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d0b00-a4d4-4c4e-abbe-52ec2aa70c16 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.020519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 773/1549] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.021780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e11bf-ac60-4537-aa8c-36f43d06cc5b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.022061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e11bf-ac60-4537-aa8c-36f43d06cc5b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.022375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e11bf-ac60-4537-aa8c-36f43d06cc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.041629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e59e11bf-ac60-4537-aa8c-36f43d06cc5b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.042812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e59e11bf-ac60-4537-aa8c-36f43d06cc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.050630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 774/1550] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.051380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e11bf-ac60-4537-aa8c-36f43d06cc5b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.061327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e11bf-ac60-4537-aa8c-36f43d06cc5b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.062844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 777/1551] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:39 2022] POST /identity/v3/users => generated 284 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:40.064208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16899b-6d3d-4094-bc4d-2059981345ba None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.064559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16899b-6d3d-4094-bc4d-2059981345ba None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.064902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16899b-6d3d-4094-bc4d-2059981345ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.068974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161328e-cdcc-4850-92e1-02003f655d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.069892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161328e-cdcc-4850-92e1-02003f655d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.070177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161328e-cdcc-4850-92e1-02003f655d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.070454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161328e-cdcc-4850-92e1-02003f655d0e admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.071105 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6161328e-cdcc-4850-92e1-02003f655d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.073028 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6161328e-cdcc-4850-92e1-02003f655d0e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.085400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6c16899b-6d3d-4094-bc4d-2059981345ba None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.086691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6c16899b-6d3d-4094-bc4d-2059981345ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.094734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 778/1552] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.095925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a3bcc-f6ee-457a-8595-646b06c367e2 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.096205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a3bcc-f6ee-457a-8595-646b06c367e2 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.096477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a3bcc-f6ee-457a-8595-646b06c367e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.116933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8c9a3bcc-f6ee-457a-8595-646b06c367e2 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.118142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8c9a3bcc-f6ee-457a-8595-646b06c367e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.126009 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 779/1553] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.127203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17110af0-4adb-43e0-976b-0a8a7b3fdf6b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.127483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17110af0-4adb-43e0-976b-0a8a7b3fdf6b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.127758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17110af0-4adb-43e0-976b-0a8a7b3fdf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.146724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-17110af0-4adb-43e0-976b-0a8a7b3fdf6b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.148161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-17110af0-4adb-43e0-976b-0a8a7b3fdf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.156122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 780/1554] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.157283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac650-ab73-4986-ad0d-66ed2e90ba06 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.157573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac650-ab73-4986-ad0d-66ed2e90ba06 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.157836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac650-ab73-4986-ad0d-66ed2e90ba06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.173038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 775/1555] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/users => generated 281 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:40.174680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76263a9-cb76-4958-bead-a4cfb683c0ee None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.174969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76263a9-cb76-4958-bead-a4cfb683c0ee None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.175267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76263a9-cb76-4958-bead-a4cfb683c0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.177224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e90ac650-ab73-4986-ad0d-66ed2e90ba06 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.178629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e90ac650-ab73-4986-ad0d-66ed2e90ba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.186574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 781/1556] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.187788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0f4d-a6ef-43ac-a25d-6f9c35f8d837 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.188071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0f4d-a6ef-43ac-a25d-6f9c35f8d837 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.188343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0f4d-a6ef-43ac-a25d-6f9c35f8d837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.200493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c76263a9-cb76-4958-bead-a4cfb683c0ee None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.201730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c76263a9-cb76-4958-bead-a4cfb683c0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.207741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-ca2b0f4d-a6ef-43ac-a25d-6f9c35f8d837 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.208928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ca2b0f4d-a6ef-43ac-a25d-6f9c35f8d837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.210895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 776/1557] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.211804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76263a9-cb76-4958-bead-a4cfb683c0ee None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.220769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 782/1558] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.221047 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76263a9-cb76-4958-bead-a4cfb683c0ee None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.221745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b0f4d-a6ef-43ac-a25d-6f9c35f8d837 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.226044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001fe7c-4ea2-41a5-a7a9-77ce04a290dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.226935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fe7c-4ea2-41a5-a7a9-77ce04a290dd admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.226935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fe7c-4ea2-41a5-a7a9-77ce04a290dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.227301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fe7c-4ea2-41a5-a7a9-77ce04a290dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.227901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001fe7c-4ea2-41a5-a7a9-77ce04a290dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.229543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001fe7c-4ea2-41a5-a7a9-77ce04a290dd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.233783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b0f4d-a6ef-43ac-a25d-6f9c35f8d837 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.238449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099810a-1c54-47fc-82ca-3a59c9c65dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.239160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099810a-1c54-47fc-82ca-3a59c9c65dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.239509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099810a-1c54-47fc-82ca-3a59c9c65dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.239851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099810a-1c54-47fc-82ca-3a59c9c65dfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.240299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 777/1559] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:40.241086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099810a-1c54-47fc-82ca-3a59c9c65dfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:40.241392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099810a-1c54-47fc-82ca-3a59c9c65dfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.241836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ce962-fd5a-4629-8861-1b1b8e70bf4c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.242115 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ce962-fd5a-4629-8861-1b1b8e70bf4c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.242386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ce962-fd5a-4629-8861-1b1b8e70bf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.243196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099810a-1c54-47fc-82ca-3a59c9c65dfd admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.255656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 783/1560] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:40.257847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa2e19-9621-4bcb-936d-68e945efdb32 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.258131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa2e19-9621-4bcb-936d-68e945efdb32 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.258673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa2e19-9621-4bcb-936d-68e945efdb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.262926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4e7ce962-fd5a-4629-8861-1b1b8e70bf4c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.264227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4e7ce962-fd5a-4629-8861-1b1b8e70bf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.272240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 778/1561] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.273017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ce962-fd5a-4629-8861-1b1b8e70bf4c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.281414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-90fa2e19-9621-4bcb-936d-68e945efdb32 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.282522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ce962-fd5a-4629-8861-1b1b8e70bf4c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.282678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-90fa2e19-9621-4bcb-936d-68e945efdb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.287097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b28a2f-865e-43f4-9338-c484d04481df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.287707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28a2f-865e-43f4-9338-c484d04481df admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.287984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28a2f-865e-43f4-9338-c484d04481df admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.288344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28a2f-865e-43f4-9338-c484d04481df admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.288981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b28a2f-865e-43f4-9338-c484d04481df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.290543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b28a2f-865e-43f4-9338-c484d04481df admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.290740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 784/1562] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.291970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d6a00-67f9-44c4-809f-54eff2802c78 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.292252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d6a00-67f9-44c4-809f-54eff2802c78 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.292515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d6a00-67f9-44c4-809f-54eff2802c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.293369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:40.301791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 779/1563] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:40.305811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b28a2f-865e-43f4-9338-c484d04481df admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.313039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b28a2f-865e-43f4-9338-c484d04481df admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.314049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-307d6a00-67f9-44c4-809f-54eff2802c78 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:40.315285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-307d6a00-67f9-44c4-809f-54eff2802c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:40.319783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.320378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.320659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.320950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.321576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:40.321880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.323632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.327937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 785/1564] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:40.328742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d6a00-67f9-44c4-809f-54eff2802c78 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.344522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 780/1565] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:40.344522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.344817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d6a00-67f9-44c4-809f-54eff2802c78 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.344817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.344817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.344817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.345228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.345228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:40.345597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.347385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.348139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71304115-a837-4e10-92b4-014ab28659eb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.354227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928470-e073-403f-bf00-4bb4a5bc15c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.354990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928470-e073-403f-bf00-4bb4a5bc15c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.355298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928470-e073-403f-bf00-4bb4a5bc15c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.355566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928470-e073-403f-bf00-4bb4a5bc15c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.356197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928470-e073-403f-bf00-4bb4a5bc15c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.357765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928470-e073-403f-bf00-4bb4a5bc15c2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.359789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 786/1566] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:40.360573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.368787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 781/1567] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:40.369638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928470-e073-403f-bf00-4bb4a5bc15c2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.370519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc848ab-8f23-4ea8-886f-15642bfd2614 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.375704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238842-7c5d-473a-9101-36f071b70e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.376300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238842-7c5d-473a-9101-36f071b70e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.376580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238842-7c5d-473a-9101-36f071b70e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.376856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238842-7c5d-473a-9101-36f071b70e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.377605 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238842-7c5d-473a-9101-36f071b70e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.379616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238842-7c5d-473a-9101-36f071b70e56 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.380219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928470-e073-403f-bf00-4bb4a5bc15c2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.385393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.386065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.386350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.386625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.387301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:40.387606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.389213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.394708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:40.402378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 782/1568] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:40.408935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.412774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06228960-59a9-494a-bdde-50ce004c8879 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.417554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.418132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.418410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.418681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.419343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:40.419643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.421278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.433375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 783/1569] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:40.434161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.444840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a8aea-5bd0-44d9-92c7-10708edff048 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.449362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabecbe0-6e87-4bbf-a6ec-c4d7785f2692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.449934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabecbe0-6e87-4bbf-a6ec-c4d7785f2692 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.450295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabecbe0-6e87-4bbf-a6ec-c4d7785f2692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.450551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabecbe0-6e87-4bbf-a6ec-c4d7785f2692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.451210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabecbe0-6e87-4bbf-a6ec-c4d7785f2692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.452773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabecbe0-6e87-4bbf-a6ec-c4d7785f2692 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.463084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 784/1570] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:40.463848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabecbe0-6e87-4bbf-a6ec-c4d7785f2692 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.473150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabecbe0-6e87-4bbf-a6ec-c4d7785f2692 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.474620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 787/1571] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:40.475993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238842-7c5d-473a-9101-36f071b70e56 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.533487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.534064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.534272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.534586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.535261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:40.535571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.537160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.548947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238842-7c5d-473a-9101-36f071b70e56 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.556959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 785/1572] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:40.556959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.557239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb2a84-9d8c-4a52-b2d7-f99b2084dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.557239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb2a84-9d8c-4a52-b2d7-f99b2084dbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.557239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb2a84-9d8c-4a52-b2d7-f99b2084dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.557239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb2a84-9d8c-4a52-b2d7-f99b2084dbb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.557682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb2a84-9d8c-4a52-b2d7-f99b2084dbb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.557682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb2a84-9d8c-4a52-b2d7-f99b2084dbb8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.566620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d866d2-993c-4192-b904-9fdac6a7e16e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.571588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb11bb3-2a3f-4b9b-942d-a966ea27b066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.572187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11bb3-2a3f-4b9b-942d-a966ea27b066 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.572484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11bb3-2a3f-4b9b-942d-a966ea27b066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.572739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11bb3-2a3f-4b9b-942d-a966ea27b066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.573358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb11bb3-2a3f-4b9b-942d-a966ea27b066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.575305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb11bb3-2a3f-4b9b-942d-a966ea27b066 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.635237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 788/1573] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/projects => generated 393 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:40.636194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb2a84-9d8c-4a52-b2d7-f99b2084dbb8 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.660414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 786/1574] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:40.660414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb11bb3-2a3f-4b9b-942d-a966ea27b066 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.702104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb2a84-9d8c-4a52-b2d7-f99b2084dbb8 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.721158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb11bb3-2a3f-4b9b-942d-a966ea27b066 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.726388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8ce74-e980-4176-af14-b9991c0faf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.726983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8ce74-e980-4176-af14-b9991c0faf86 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.727284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8ce74-e980-4176-af14-b9991c0faf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.727555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8ce74-e980-4176-af14-b9991c0faf86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.728163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8ce74-e980-4176-af14-b9991c0faf86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.730037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8ce74-e980-4176-af14-b9991c0faf86 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.745319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:40.758044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8dc9e-66c7-46f1-914f-dc4edcb64318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.759017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dc9e-66c7-46f1-914f-dc4edcb64318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.759307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dc9e-66c7-46f1-914f-dc4edcb64318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.759584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dc9e-66c7-46f1-914f-dc4edcb64318 admin admin] PATH_INFO: `/v3/projects/33cefdfac4ed4cc4b241301596ce4cd8/users/415a32d179144aeb97aff3ea81835835/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.774023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8dc9e-66c7-46f1-914f-dc4edcb64318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cefdfac4ed4cc4b241301596ce4cd8, user_id=415a32d179144aeb97aff3ea81835835, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.776738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8dc9e-66c7-46f1-914f-dc4edcb64318 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.860483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 789/1575] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:40 2022] PUT /identity/v3/projects/33cefdfac4ed4cc4b241301596ce4cd8/users/415a32d179144aeb97aff3ea81835835/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:40.860483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8dc9e-66c7-46f1-914f-dc4edcb64318 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.860483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:40.865568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 787/1576] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/projects => generated 403 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:40.865568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8ce74-e980-4176-af14-b9991c0faf86 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:40.913123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8dc9e-66c7-46f1-914f-dc4edcb64318 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.928724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8ce74-e980-4176-af14-b9991c0faf86 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:40.933336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.933989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.934268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.934539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] PATH_INFO: `/v3/projects/d92481181f434ae49c232fd807f4b896/users/3d1b8b586b4949109e468eb3c744b63c/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.951390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92481181f434ae49c232fd807f4b896, user_id=3d1b8b586b4949109e468eb3c744b63c, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.952622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:21:40.952622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Aug 07 12:21:40.952622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: Aug 07 12:21:40.954540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:40.969272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029217aa-775f-4ba5-a1ee-875fbdc2d1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:40.969873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029217aa-775f-4ba5-a1ee-875fbdc2d1ee admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:40.970149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029217aa-775f-4ba5-a1ee-875fbdc2d1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:40.970419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029217aa-775f-4ba5-a1ee-875fbdc2d1ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:40.971065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029217aa-775f-4ba5-a1ee-875fbdc2d1ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:40.972956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029217aa-775f-4ba5-a1ee-875fbdc2d1ee admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.026532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 788/1577] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:40 2022] PUT /identity/v3/projects/d92481181f434ae49c232fd807f4b896/users/3d1b8b586b4949109e468eb3c744b63c/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:41.027376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.057507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 790/1578] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:40 2022] POST /identity/v3/projects => generated 389 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.058534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029217aa-775f-4ba5-a1ee-875fbdc2d1ee admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.093066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fffac9-1b70-4e79-b990-f2abc20849f8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.121477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029217aa-775f-4ba5-a1ee-875fbdc2d1ee admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.126704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0802-9a3d-4fb1-aacf-8025eb7aed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.127340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0802-9a3d-4fb1-aacf-8025eb7aed78 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.127650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0802-9a3d-4fb1-aacf-8025eb7aed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.127927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0802-9a3d-4fb1-aacf-8025eb7aed78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.128548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e0802-9a3d-4fb1-aacf-8025eb7aed78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.130505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e0802-9a3d-4fb1-aacf-8025eb7aed78 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.155959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed63681-4373-4e46-ba03-63ad66a6950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.161674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63681-4373-4e46-ba03-63ad66a6950c admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.161674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63681-4373-4e46-ba03-63ad66a6950c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.161674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63681-4373-4e46-ba03-63ad66a6950c admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.161674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed63681-4373-4e46-ba03-63ad66a6950c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.161674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed63681-4373-4e46-ba03-63ad66a6950c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.240116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 791/1579] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.241660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0802-9a3d-4fb1-aacf-8025eb7aed78 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.253204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 789/1580] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.253204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed63681-4373-4e46-ba03-63ad66a6950c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.262457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed63681-4373-4e46-ba03-63ad66a6950c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.271606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:41.276364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:41.304767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0802-9a3d-4fb1-aacf-8025eb7aed78 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.309530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c9604-cf57-4a43-b05b-e40288e0e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.310173 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9604-cf57-4a43-b05b-e40288e0e24c admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.310450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9604-cf57-4a43-b05b-e40288e0e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.310721 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9604-cf57-4a43-b05b-e40288e0e24c admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.311374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c9604-cf57-4a43-b05b-e40288e0e24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.313289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c9604-cf57-4a43-b05b-e40288e0e24c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.318295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffc2ae-1ebf-4de5-b040-1c6c6d8b5cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.319762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffc2ae-1ebf-4de5-b040-1c6c6d8b5cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.320584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffc2ae-1ebf-4de5-b040-1c6c6d8b5cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.320908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffc2ae-1ebf-4de5-b040-1c6c6d8b5cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.321525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffc2ae-1ebf-4de5-b040-1c6c6d8b5cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.323467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffc2ae-1ebf-4de5-b040-1c6c6d8b5cc5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.427557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 792/1581] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.429388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0925820-f420-4b70-a738-342ff2fbe4a0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.429672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0925820-f420-4b70-a738-342ff2fbe4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.429949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0925820-f420-4b70-a738-342ff2fbe4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.454152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 790/1582] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.454152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffc2ae-1ebf-4de5-b040-1c6c6d8b5cc5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.457846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a0925820-f420-4b70-a738-342ff2fbe4a0 None None] MFA Rules not processed for user `415a32d179144aeb97aff3ea81835835`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:41.459709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a0925820-f420-4b70-a738-342ff2fbe4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:41.461969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffc2ae-1ebf-4de5-b040-1c6c6d8b5cc5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.466645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2939a85-c6c1-4524-8d48-0b128b24a899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.467383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2939a85-c6c1-4524-8d48-0b128b24a899 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.467802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2939a85-c6c1-4524-8d48-0b128b24a899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.468081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2939a85-c6c1-4524-8d48-0b128b24a899 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.468698 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2939a85-c6c1-4524-8d48-0b128b24a899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.470691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2939a85-c6c1-4524-8d48-0b128b24a899 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.525187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 793/1583] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:41.526445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08772d-578c-49bf-8fe0-029e94429559 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.526755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08772d-578c-49bf-8fe0-029e94429559 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.527065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08772d-578c-49bf-8fe0-029e94429559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.552911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 791/1584] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.554306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2939a85-c6c1-4524-8d48-0b128b24a899 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.555160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-fd08772d-578c-49bf-8fe0-029e94429559 None None] MFA Rules not processed for user `3d1b8b586b4949109e468eb3c744b63c`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:41.556666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-fd08772d-578c-49bf-8fe0-029e94429559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:41.564455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2939a85-c6c1-4524-8d48-0b128b24a899 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.569547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e80c-0201-4ddd-8cab-f7d0bdf0b1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.570173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e80c-0201-4ddd-8cab-f7d0bdf0b1bd admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.570454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e80c-0201-4ddd-8cab-f7d0bdf0b1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.570900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e80c-0201-4ddd-8cab-f7d0bdf0b1bd admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.571372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e80c-0201-4ddd-8cab-f7d0bdf0b1bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.573276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e80c-0201-4ddd-8cab-f7d0bdf0b1bd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.620385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 794/1585] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:41.621199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08772d-578c-49bf-8fe0-029e94429559 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.631838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08772d-578c-49bf-8fe0-029e94429559 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.636871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578f6b3-54e9-4145-a37a-1deb1588559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.637468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578f6b3-54e9-4145-a37a-1deb1588559c admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.637751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578f6b3-54e9-4145-a37a-1deb1588559c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.638025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578f6b3-54e9-4145-a37a-1deb1588559c admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.638643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578f6b3-54e9-4145-a37a-1deb1588559c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.642799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 792/1586] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/users => generated 295 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.646753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578f6b3-54e9-4145-a37a-1deb1588559c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.647432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e80c-0201-4ddd-8cab-f7d0bdf0b1bd admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.653186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e80c-0201-4ddd-8cab-f7d0bdf0b1bd admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.657849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48771c62-ec16-4e65-9970-c23d53f7f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.658446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771c62-ec16-4e65-9970-c23d53f7f37f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.658726 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771c62-ec16-4e65-9970-c23d53f7f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.659032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771c62-ec16-4e65-9970-c23d53f7f37f admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.659815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48771c62-ec16-4e65-9970-c23d53f7f37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.661395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48771c62-ec16-4e65-9970-c23d53f7f37f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.671589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 793/1587] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.671589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48771c62-ec16-4e65-9970-c23d53f7f37f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.681692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48771c62-ec16-4e65-9970-c23d53f7f37f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.686091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b5cd0-d601-413e-967b-24e1c7c88c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.686647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b5cd0-d601-413e-967b-24e1c7c88c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.686923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b5cd0-d601-413e-967b-24e1c7c88c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.687232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b5cd0-d601-413e-967b-24e1c7c88c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.687872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b5cd0-d601-413e-967b-24e1c7c88c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.689432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b5cd0-d601-413e-967b-24e1c7c88c26 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.699660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 794/1588] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.700429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b5cd0-d601-413e-967b-24e1c7c88c26 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.709542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b5cd0-d601-413e-967b-24e1c7c88c26 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.714072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fe687-85d2-443d-b1cf-f6cad6c9fdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.714619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fe687-85d2-443d-b1cf-f6cad6c9fdb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.714897 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fe687-85d2-443d-b1cf-f6cad6c9fdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.715206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fe687-85d2-443d-b1cf-f6cad6c9fdb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.715817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173fe687-85d2-443d-b1cf-f6cad6c9fdb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.717358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173fe687-85d2-443d-b1cf-f6cad6c9fdb8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.727882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 795/1589] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.728704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fe687-85d2-443d-b1cf-f6cad6c9fdb8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.736456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 795/1590] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.737309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578f6b3-54e9-4145-a37a-1deb1588559c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.747780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578f6b3-54e9-4145-a37a-1deb1588559c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.752433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9fcdf-764e-4acd-8c0d-b49056681871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.753016 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9fcdf-764e-4acd-8c0d-b49056681871 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.753293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9fcdf-764e-4acd-8c0d-b49056681871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.753567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9fcdf-764e-4acd-8c0d-b49056681871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.754203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9fcdf-764e-4acd-8c0d-b49056681871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.755840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9fcdf-764e-4acd-8c0d-b49056681871 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.766404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 796/1591] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.767198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9fcdf-764e-4acd-8c0d-b49056681871 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.776914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9fcdf-764e-4acd-8c0d-b49056681871 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.781382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8b425-c60c-4c7d-be11-3eb609969f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.781950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b425-c60c-4c7d-be11-3eb609969f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.788019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b425-c60c-4c7d-be11-3eb609969f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.788019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b425-c60c-4c7d-be11-3eb609969f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.788019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8b425-c60c-4c7d-be11-3eb609969f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.788019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8b425-c60c-4c7d-be11-3eb609969f4c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.788495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fe687-85d2-443d-b1cf-f6cad6c9fdb8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.790074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a53e12-a808-413c-967f-b1b9c02f3e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.790550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:41.790810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a53e12-a808-413c-967f-b1b9c02f3e46 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.791191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a53e12-a808-413c-967f-b1b9c02f3e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.791554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a53e12-a808-413c-967f-b1b9c02f3e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.800075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a53e12-a808-413c-967f-b1b9c02f3e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.801687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a53e12-a808-413c-967f-b1b9c02f3e46 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.810494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 796/1592] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:41.813067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a53e12-a808-413c-967f-b1b9c02f3e46 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.821880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a53e12-a808-413c-967f-b1b9c02f3e46 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.826487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635675-a3ae-47c2-b181-d6915f5cad7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.827098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635675-a3ae-47c2-b181-d6915f5cad7c admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.827381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635675-a3ae-47c2-b181-d6915f5cad7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.827688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635675-a3ae-47c2-b181-d6915f5cad7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.828328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635675-a3ae-47c2-b181-d6915f5cad7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.829894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635675-a3ae-47c2-b181-d6915f5cad7c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.841019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 797/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.841871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635675-a3ae-47c2-b181-d6915f5cad7c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.851555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635675-a3ae-47c2-b181-d6915f5cad7c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.856109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1d7d-d25a-4666-b8e2-a23c58321a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.856675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1d7d-d25a-4666-b8e2-a23c58321a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.856960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1d7d-d25a-4666-b8e2-a23c58321a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.857235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1d7d-d25a-4666-b8e2-a23c58321a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.857860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f1d7d-d25a-4666-b8e2-a23c58321a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.859460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f1d7d-d25a-4666-b8e2-a23c58321a9a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.869282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 797/1594] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:41 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:41.870209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 798/1595] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.872754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8b425-c60c-4c7d-be11-3eb609969f4c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.872857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1d7d-d25a-4666-b8e2-a23c58321a9a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.884654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1d7d-d25a-4666-b8e2-a23c58321a9a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.897520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:41.940757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8b425-c60c-4c7d-be11-3eb609969f4c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.946363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614088a4-5211-4c4f-b8a1-a11abc239818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.946977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614088a4-5211-4c4f-b8a1-a11abc239818 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.947418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614088a4-5211-4c4f-b8a1-a11abc239818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.947697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614088a4-5211-4c4f-b8a1-a11abc239818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.948353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614088a4-5211-4c4f-b8a1-a11abc239818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.950139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614088a4-5211-4c4f-b8a1-a11abc239818 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.956278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c14d-b5a0-49f1-8f8e-58a061b5d6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.957034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c14d-b5a0-49f1-8f8e-58a061b5d6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.959209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c14d-b5a0-49f1-8f8e-58a061b5d6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.959209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c14d-b5a0-49f1-8f8e-58a061b5d6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.959209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56c14d-b5a0-49f1-8f8e-58a061b5d6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.960423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56c14d-b5a0-49f1-8f8e-58a061b5d6a3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.965314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 798/1596] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.965314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614088a4-5211-4c4f-b8a1-a11abc239818 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.974225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614088a4-5211-4c4f-b8a1-a11abc239818 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.976965 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 799/1597] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.979961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da2380-5b2d-4e5d-b9b3-27f2b7e2a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:41.980343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c14d-b5a0-49f1-8f8e-58a061b5d6a3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:41.980818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2380-5b2d-4e5d-b9b3-27f2b7e2a8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:41.981201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2380-5b2d-4e5d-b9b3-27f2b7e2a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:41.981606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2380-5b2d-4e5d-b9b3-27f2b7e2a8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:41.982356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da2380-5b2d-4e5d-b9b3-27f2b7e2a8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:41.984128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da2380-5b2d-4e5d-b9b3-27f2b7e2a8c2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:41.992317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c14d-b5a0-49f1-8f8e-58a061b5d6a3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:41.995639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 799/1598] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:41.996546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da2380-5b2d-4e5d-b9b3-27f2b7e2a8c2 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.000695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061de05-5f93-4f0e-a9a7-3cea37264992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.001304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061de05-5f93-4f0e-a9a7-3cea37264992 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.001645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061de05-5f93-4f0e-a9a7-3cea37264992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.001956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061de05-5f93-4f0e-a9a7-3cea37264992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.002604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061de05-5f93-4f0e-a9a7-3cea37264992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.004222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061de05-5f93-4f0e-a9a7-3cea37264992 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.008148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da2380-5b2d-4e5d-b9b3-27f2b7e2a8c2 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.013334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08d457-8dda-4031-84b7-da3fdbb56653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.014042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d457-8dda-4031-84b7-da3fdbb56653 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.014444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d457-8dda-4031-84b7-da3fdbb56653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.014824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d457-8dda-4031-84b7-da3fdbb56653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.015609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08d457-8dda-4031-84b7-da3fdbb56653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.016684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 800/1599] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:42.017315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061de05-5f93-4f0e-a9a7-3cea37264992 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.020522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08d457-8dda-4031-84b7-da3fdbb56653 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.027652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061de05-5f93-4f0e-a9a7-3cea37264992 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.032575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde301d6-bb00-41e1-b14d-9add725e1790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.033160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde301d6-bb00-41e1-b14d-9add725e1790 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.033442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde301d6-bb00-41e1-b14d-9add725e1790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.033715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde301d6-bb00-41e1-b14d-9add725e1790 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.034232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 800/1600] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:42.034492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde301d6-bb00-41e1-b14d-9add725e1790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.035130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08d457-8dda-4031-84b7-da3fdbb56653 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.036558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde301d6-bb00-41e1-b14d-9add725e1790 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.045778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08d457-8dda-4031-84b7-da3fdbb56653 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.050767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da03805-1d61-479c-a3bc-78e71e76f722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.051560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03805-1d61-479c-a3bc-78e71e76f722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.051919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03805-1d61-479c-a3bc-78e71e76f722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.052299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03805-1d61-479c-a3bc-78e71e76f722 admin admin] PATH_INFO: `/v3/projects/4463509e9402446f93dd4334865e5b36/users/f68c2c20add245868f00e405885fac16/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.071781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da03805-1d61-479c-a3bc-78e71e76f722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4463509e9402446f93dd4334865e5b36, user_id=f68c2c20add245868f00e405885fac16, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.074485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da03805-1d61-479c-a3bc-78e71e76f722 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.114280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 801/1601] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:42 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:42.115281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde301d6-bb00-41e1-b14d-9add725e1790 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.125705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde301d6-bb00-41e1-b14d-9add725e1790 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.130387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a139fcd-9ffd-4966-b773-c4ffd1355850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.131137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139fcd-9ffd-4966-b773-c4ffd1355850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.131922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139fcd-9ffd-4966-b773-c4ffd1355850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.132199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139fcd-9ffd-4966-b773-c4ffd1355850 admin admin] PATH_INFO: `/v3/projects/bd31c40f797549d082e083324e285b6e/users/90dc7b751dd14c6f80205e709826423b/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.138092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 801/1602] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:42 2022] PUT /identity/v3/projects/4463509e9402446f93dd4334865e5b36/users/f68c2c20add245868f00e405885fac16/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:42.139116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da03805-1d61-479c-a3bc-78e71e76f722 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.160315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a139fcd-9ffd-4966-b773-c4ffd1355850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd31c40f797549d082e083324e285b6e, user_id=90dc7b751dd14c6f80205e709826423b, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.163708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a139fcd-9ffd-4966-b773-c4ffd1355850 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.202191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da03805-1d61-479c-a3bc-78e71e76f722 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.206883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae33cd-90b9-4e9f-84b4-f2108ed80764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.207615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae33cd-90b9-4e9f-84b4-f2108ed80764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.207935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae33cd-90b9-4e9f-84b4-f2108ed80764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.208207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae33cd-90b9-4e9f-84b4-f2108ed80764 admin admin] PATH_INFO: `/v3/projects/71ef7979b9fc4098a9219118616fb096/users/9e9d3a02e1c8420ca1993eedc5009601/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.223024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae33cd-90b9-4e9f-84b4-f2108ed80764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ef7979b9fc4098a9219118616fb096, user_id=9e9d3a02e1c8420ca1993eedc5009601, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.226286 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae33cd-90b9-4e9f-84b4-f2108ed80764 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.234238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 802/1603] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:42 2022] PUT /identity/v3/projects/bd31c40f797549d082e083324e285b6e/users/90dc7b751dd14c6f80205e709826423b/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:42.235096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a139fcd-9ffd-4966-b773-c4ffd1355850 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.297203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a139fcd-9ffd-4966-b773-c4ffd1355850 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.301955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70776f61-f29b-4a0e-a766-b23e2c18dd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.302549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776f61-f29b-4a0e-a766-b23e2c18dd3b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.302926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776f61-f29b-4a0e-a766-b23e2c18dd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.303251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776f61-f29b-4a0e-a766-b23e2c18dd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.304058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70776f61-f29b-4a0e-a766-b23e2c18dd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.305776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70776f61-f29b-4a0e-a766-b23e2c18dd3b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.315743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 803/1604] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:42.316752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70776f61-f29b-4a0e-a766-b23e2c18dd3b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.321689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 802/1605] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:42 2022] PUT /identity/v3/projects/71ef7979b9fc4098a9219118616fb096/users/9e9d3a02e1c8420ca1993eedc5009601/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:42.322631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae33cd-90b9-4e9f-84b4-f2108ed80764 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.337065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70776f61-f29b-4a0e-a766-b23e2c18dd3b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.343810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:42.387201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae33cd-90b9-4e9f-84b4-f2108ed80764 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.388782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:42.392090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056a235-922e-4a0e-96a2-464ab362520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.393023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a235-922e-4a0e-96a2-464ab362520a admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.393355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a235-922e-4a0e-96a2-464ab362520a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.393646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a235-922e-4a0e-96a2-464ab362520a admin admin] PATH_INFO: `/v3/projects/547ebf661b1b48f289d734a71b494889/users/c8fb20cfc3b6479198059b4383f47527/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.395271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34304f30-099e-4483-b784-6130f6882842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.395924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34304f30-099e-4483-b784-6130f6882842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.396200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34304f30-099e-4483-b784-6130f6882842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.396471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34304f30-099e-4483-b784-6130f6882842 admin admin] PATH_INFO: `/v3/projects/5c15466dfd944d929c9e5db6f9f144bb/users/461eb7f2caaf4854b142ff70666a82a9/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.411688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34304f30-099e-4483-b784-6130f6882842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c15466dfd944d929c9e5db6f9f144bb, user_id=461eb7f2caaf4854b142ff70666a82a9, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.412532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056a235-922e-4a0e-96a2-464ab362520a admin admin] RBAC: Authorizing `identity:create_grant(project_id=547ebf661b1b48f289d734a71b494889, user_id=c8fb20cfc3b6479198059b4383f47527, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.414791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34304f30-099e-4483-b784-6130f6882842 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.415690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056a235-922e-4a0e-96a2-464ab362520a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.502040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 804/1607] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:42 2022] PUT /identity/v3/projects/5c15466dfd944d929c9e5db6f9f144bb/users/461eb7f2caaf4854b142ff70666a82a9/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:42.504541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34304f30-099e-4483-b784-6130f6882842 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.513515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 803/1607] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:42 2022] PUT /identity/v3/projects/547ebf661b1b48f289d734a71b494889/users/c8fb20cfc3b6479198059b4383f47527/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:42.515095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef766a2-c636-482e-bd04-b65d61ec2301 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.515389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef766a2-c636-482e-bd04-b65d61ec2301 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.515668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef766a2-c636-482e-bd04-b65d61ec2301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.543128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-aef766a2-c636-482e-bd04-b65d61ec2301 None None] MFA Rules not processed for user `f68c2c20add245868f00e405885fac16`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:42.544347 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-aef766a2-c636-482e-bd04-b65d61ec2301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:42.567916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34304f30-099e-4483-b784-6130f6882842 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.623857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 804/1608] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:42.632252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56330df-b765-404c-a35a-2d845ca25cd4 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.632252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56330df-b765-404c-a35a-2d845ca25cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.632252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56330df-b765-404c-a35a-2d845ca25cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.632526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a5ccd-1fea-4d4a-b1f1-c5c62e58124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.632526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a5ccd-1fea-4d4a-b1f1-c5c62e58124b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.632526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a5ccd-1fea-4d4a-b1f1-c5c62e58124b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.632526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a5ccd-1fea-4d4a-b1f1-c5c62e58124b admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.635243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a5ccd-1fea-4d4a-b1f1-c5c62e58124b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.637546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a5ccd-1fea-4d4a-b1f1-c5c62e58124b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.650330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 805/1609] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:42.651436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a5ccd-1fea-4d4a-b1f1-c5c62e58124b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.653288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c56330df-b765-404c-a35a-2d845ca25cd4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:42.654858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c56330df-b765-404c-a35a-2d845ca25cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:42.663648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a5ccd-1fea-4d4a-b1f1-c5c62e58124b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.667463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 805/1610] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:42.670669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19600f-19cc-4684-97a4-17b4e5075a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.671441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19600f-19cc-4684-97a4-17b4e5075a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.671728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19600f-19cc-4684-97a4-17b4e5075a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.671843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18dd6-994f-45e7-afb8-4ed818e8a864 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.672005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19600f-19cc-4684-97a4-17b4e5075a99 admin admin] PATH_INFO: `/v3/projects/5becdfe9ba3b48d5b11668d6eeec33a4/users/5401baffbd794016b01d109e8e2529ac/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.672291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18dd6-994f-45e7-afb8-4ed818e8a864 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.673573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18dd6-994f-45e7-afb8-4ed818e8a864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.690139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19600f-19cc-4684-97a4-17b4e5075a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5becdfe9ba3b48d5b11668d6eeec33a4, user_id=5401baffbd794016b01d109e8e2529ac, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.692827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19600f-19cc-4684-97a4-17b4e5075a99 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.714425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4ad18dd6-994f-45e7-afb8-4ed818e8a864 None None] MFA Rules not processed for user `9e9d3a02e1c8420ca1993eedc5009601`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:42.717928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4ad18dd6-994f-45e7-afb8-4ed818e8a864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:42.760370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 806/1611] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:42 2022] PUT /identity/v3/projects/5becdfe9ba3b48d5b11668d6eeec33a4/users/5401baffbd794016b01d109e8e2529ac/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:42.760370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7199076-f30d-494e-8395-b56fad127ed8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.760370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7199076-f30d-494e-8395-b56fad127ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.760370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7199076-f30d-494e-8395-b56fad127ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.775337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f7199076-f30d-494e-8395-b56fad127ed8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:42.776643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f7199076-f30d-494e-8395-b56fad127ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:42.795910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 806/1612] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:42.797347 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3971b-b2df-4bdc-b348-e0a1dcaaf667 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.797634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3971b-b2df-4bdc-b348-e0a1dcaaf667 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.797902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3971b-b2df-4bdc-b348-e0a1dcaaf667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.827524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:42.830646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f7e3971b-b2df-4bdc-b348-e0a1dcaaf667 None None] MFA Rules not processed for user `461eb7f2caaf4854b142ff70666a82a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:42.831834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f7e3971b-b2df-4bdc-b348-e0a1dcaaf667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:42.838648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:42.847714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 807/1613] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:42.848519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7199076-f30d-494e-8395-b56fad127ed8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.858122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7199076-f30d-494e-8395-b56fad127ed8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.862860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5b4d-13dc-448c-8c18-3c403caf763a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:42.863549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5b4d-13dc-448c-8c18-3c403caf763a admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.864445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5b4d-13dc-448c-8c18-3c403caf763a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.864445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5b4d-13dc-448c-8c18-3c403caf763a admin admin] PATH_INFO: `/v3/projects/4c4f5478edba4618864db2664e93c0c6/users/0c30712dfe124f1282befda8038f216a/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.879519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5b4d-13dc-448c-8c18-3c403caf763a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c4f5478edba4618864db2664e93c0c6, user_id=0c30712dfe124f1282befda8038f216a, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:42.883265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5b4d-13dc-448c-8c18-3c403caf763a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:42.903535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 807/1614] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:42.904580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3971b-b2df-4bdc-b348-e0a1dcaaf667 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:42.905779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f7e3971b-b2df-4bdc-b348-e0a1dcaaf667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:42.960485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 808/1615] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:42 2022] PUT /identity/v3/projects/4c4f5478edba4618864db2664e93c0c6/users/0c30712dfe124f1282befda8038f216a/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:42.961987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3971b-b2df-4bdc-b348-e0a1dcaaf667 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:42.962101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ebec2-6d2c-41ba-b13a-3ef72acbc0ed None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.962101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ebec2-6d2c-41ba-b13a-3ef72acbc0ed None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.962261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ebec2-6d2c-41ba-b13a-3ef72acbc0ed None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.963498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 809/1616] 10.222.0.30 () {54 vars in 962 bytes} [Sun Aug 7 12:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:21:42.964503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20b79f-8fa9-46c6-9bfe-db8e8230b024 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:42.964783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20b79f-8fa9-46c6-9bfe-db8e8230b024 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:42.965055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20b79f-8fa9-46c6-9bfe-db8e8230b024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:42.984336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9c20b79f-8fa9-46c6-9bfe-db8e8230b024 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:42.990149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9c20b79f-8fa9-46c6-9bfe-db8e8230b024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:43.012859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2724c6e-5d1f-4efe-b4b7-891d5a847ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.013370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2724c6e-5d1f-4efe-b4b7-891d5a847ade service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.013653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2724c6e-5d1f-4efe-b4b7-891d5a847ade service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.013952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2724c6e-5d1f-4efe-b4b7-891d5a847ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.045736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 810/1617] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:43.046825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20b79f-8fa9-46c6-9bfe-db8e8230b024 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.057593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20b79f-8fa9-46c6-9bfe-db8e8230b024 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.063727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b8453-f615-4920-b829-c5a243ded033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.064373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b8453-f615-4920-b829-c5a243ded033 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.064652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b8453-f615-4920-b829-c5a243ded033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.064930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b8453-f615-4920-b829-c5a243ded033 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.065551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b8453-f615-4920-b829-c5a243ded033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.067481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b8453-f615-4920-b829-c5a243ded033 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.076693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2724c6e-5d1f-4efe-b4b7-891d5a847ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.081211 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2724c6e-5d1f-4efe-b4b7-891d5a847ade service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.087928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 808/1618] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:43.088767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2724c6e-5d1f-4efe-b4b7-891d5a847ade service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.096832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2724c6e-5d1f-4efe-b4b7-891d5a847ade service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.102361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4dfe3-949f-442a-b731-b61f1785aeb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.107348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4dfe3-949f-442a-b731-b61f1785aeb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.107348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4dfe3-949f-442a-b731-b61f1785aeb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.107348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4dfe3-949f-442a-b731-b61f1785aeb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.117782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4dfe3-949f-442a-b731-b61f1785aeb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.119261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4dfe3-949f-442a-b731-b61f1785aeb2 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.134498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 809/1619] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:43.135342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4dfe3-949f-442a-b731-b61f1785aeb2 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.144522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4dfe3-949f-442a-b731-b61f1785aeb2 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.149412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf549e1-1530-4a9d-830e-41040c56cc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.149984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf549e1-1530-4a9d-830e-41040c56cc1e admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.150272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf549e1-1530-4a9d-830e-41040c56cc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.150649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf549e1-1530-4a9d-830e-41040c56cc1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.151531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf549e1-1530-4a9d-830e-41040c56cc1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.153776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf549e1-1530-4a9d-830e-41040c56cc1e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.157553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:43.187642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 811/1620] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:43 2022] POST /identity/v3/projects => generated 391 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:43.190306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b8453-f615-4920-b829-c5a243ded033 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.253509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b8453-f615-4920-b829-c5a243ded033 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.258617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbc36c-4ea4-443a-9701-00bb7112cde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.259258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbc36c-4ea4-443a-9701-00bb7112cde7 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.259636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbc36c-4ea4-443a-9701-00bb7112cde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.259938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbc36c-4ea4-443a-9701-00bb7112cde7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.260584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbc36c-4ea4-443a-9701-00bb7112cde7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.262211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbc36c-4ea4-443a-9701-00bb7112cde7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.272779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 812/1621] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:43.274175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53436d1-3675-44a5-98f0-a9cdc44dc702 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.274468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53436d1-3675-44a5-98f0-a9cdc44dc702 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.274749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53436d1-3675-44a5-98f0-a9cdc44dc702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.290255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 810/1622] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:43 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:43.290255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf549e1-1530-4a9d-830e-41040c56cc1e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.317432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f53436d1-3675-44a5-98f0-a9cdc44dc702 None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:43.322002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f53436d1-3675-44a5-98f0-a9cdc44dc702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:43.353559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:43.353918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf549e1-1530-4a9d-830e-41040c56cc1e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.359061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322ddd-5f0f-4fce-ab2b-f0e662d2fd9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.359682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322ddd-5f0f-4fce-ab2b-f0e662d2fd9b service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.359968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322ddd-5f0f-4fce-ab2b-f0e662d2fd9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.360247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322ddd-5f0f-4fce-ab2b-f0e662d2fd9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.398049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 813/1623] 10.222.0.30 () {58 vars in 1059 bytes} [Sun Aug 7 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:43.398924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53436d1-3675-44a5-98f0-a9cdc44dc702 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.432875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df322ddd-5f0f-4fce-ab2b-f0e662d2fd9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.434429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df322ddd-5f0f-4fce-ab2b-f0e662d2fd9b service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.445411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 811/1624] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:43.446306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322ddd-5f0f-4fce-ab2b-f0e662d2fd9b service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.456675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322ddd-5f0f-4fce-ab2b-f0e662d2fd9b service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.462121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457351-2f81-45fc-b083-890335efd20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.462784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457351-2f81-45fc-b083-890335efd20c admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.463086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457351-2f81-45fc-b083-890335efd20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.463360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457351-2f81-45fc-b083-890335efd20c admin admin] PATH_INFO: `/v3/projects/4c4f5478edba4618864db2664e93c0c6/users/0c30712dfe124f1282befda8038f216a/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.465294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70457351-2f81-45fc-b083-890335efd20c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c4f5478edba4618864db2664e93c0c6, user_id=0c30712dfe124f1282befda8038f216a, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.468068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53436d1-3675-44a5-98f0-a9cdc44dc702 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.468197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70457351-2f81-45fc-b083-890335efd20c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.474036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387321de-b5e7-4f5c-bbec-3cd86a7a3271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.474676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387321de-b5e7-4f5c-bbec-3cd86a7a3271 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.475017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387321de-b5e7-4f5c-bbec-3cd86a7a3271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.475302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387321de-b5e7-4f5c-bbec-3cd86a7a3271 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.476037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387321de-b5e7-4f5c-bbec-3cd86a7a3271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.477997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387321de-b5e7-4f5c-bbec-3cd86a7a3271 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.480759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 812/1625] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:43 2022] PUT /identity/v3/projects/4c4f5478edba4618864db2664e93c0c6/users/0c30712dfe124f1282befda8038f216a/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:43.481510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457351-2f81-45fc-b083-890335efd20c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.549070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 814/1626] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:43 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:43.550113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387321de-b5e7-4f5c-bbec-3cd86a7a3271 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.562247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457351-2f81-45fc-b083-890335efd20c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.564994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387321de-b5e7-4f5c-bbec-3cd86a7a3271 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.569988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f7e75-5754-4f58-b533-0d2d685ac967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.570750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7e75-5754-4f58-b533-0d2d685ac967 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.570897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7e75-5754-4f58-b533-0d2d685ac967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.571177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7e75-5754-4f58-b533-0d2d685ac967 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.571817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f7e75-5754-4f58-b533-0d2d685ac967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.574324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f7e75-5754-4f58-b533-0d2d685ac967 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.636020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b49f6-16ac-428d-bb38-a269f6c8b7fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.636526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b49f6-16ac-428d-bb38-a269f6c8b7fd service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.636811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b49f6-16ac-428d-bb38-a269f6c8b7fd service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.637085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b49f6-16ac-428d-bb38-a269f6c8b7fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.651196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:43.660020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 813/1627] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:43 2022] POST /identity/v3/users => generated 309 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:43.661544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597737d-fd76-4d8c-8c60-e87f98fd01d4 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.662008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597737d-fd76-4d8c-8c60-e87f98fd01d4 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.662896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597737d-fd76-4d8c-8c60-e87f98fd01d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.691028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-5597737d-fd76-4d8c-8c60-e87f98fd01d4 None None] MFA Rules not processed for user `0c30712dfe124f1282befda8038f216a`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:43.692608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5597737d-fd76-4d8c-8c60-e87f98fd01d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:43.695746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:43.700024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b49f6-16ac-428d-bb38-a269f6c8b7fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.701617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b49f6-16ac-428d-bb38-a269f6c8b7fd service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.711568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 815/1628] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:43.712566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b49f6-16ac-428d-bb38-a269f6c8b7fd service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.723218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b49f6-16ac-428d-bb38-a269f6c8b7fd service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.728861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11e0dd-3fda-4aea-9b36-4ce00edd0a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.729514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e0dd-3fda-4aea-9b36-4ce00edd0a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.729791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e0dd-3fda-4aea-9b36-4ce00edd0a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.730064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e0dd-3fda-4aea-9b36-4ce00edd0a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.730754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11e0dd-3fda-4aea-9b36-4ce00edd0a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.732414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11e0dd-3fda-4aea-9b36-4ce00edd0a79 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.753235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 816/1629] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:43.754145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11e0dd-3fda-4aea-9b36-4ce00edd0a79 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.767874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11e0dd-3fda-4aea-9b36-4ce00edd0a79 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.779335 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c58af-79a0-4412-adfb-9cf645c6f206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.779896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c58af-79a0-4412-adfb-9cf645c6f206 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.780175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c58af-79a0-4412-adfb-9cf645c6f206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.780448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c58af-79a0-4412-adfb-9cf645c6f206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.781098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c58af-79a0-4412-adfb-9cf645c6f206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.782680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c58af-79a0-4412-adfb-9cf645c6f206 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.796723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 814/1630] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:43.797788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597737d-fd76-4d8c-8c60-e87f98fd01d4 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.806813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 817/1631] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:43.813893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c58af-79a0-4412-adfb-9cf645c6f206 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.831302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597737d-fd76-4d8c-8c60-e87f98fd01d4 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.833610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c58af-79a0-4412-adfb-9cf645c6f206 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:43.839445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a838-0f6f-4345-8ae7-20ea5de21d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.842387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a838-0f6f-4345-8ae7-20ea5de21d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.842768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a838-0f6f-4345-8ae7-20ea5de21d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.843178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a838-0f6f-4345-8ae7-20ea5de21d6d admin admin] PATH_INFO: `/v3/projects/6e1abf183aaf46f9beae2566b219e861/users/02b12fbbded94f4d8542c333b9b69035/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.855221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b0bd3-a227-4936-a8b6-157e2f2d3d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:43.856163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b0bd3-a227-4936-a8b6-157e2f2d3d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:43.856820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b0bd3-a227-4936-a8b6-157e2f2d3d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:43.857357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b0bd3-a227-4936-a8b6-157e2f2d3d08 admin admin] PATH_INFO: `/v3/projects/17dbc4f2888543f2adb7eb6f9ece44b7/users/9bf127c12f0348f582ba984ebc581fa8/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:43.864514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2a838-0f6f-4345-8ae7-20ea5de21d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1abf183aaf46f9beae2566b219e861, user_id=02b12fbbded94f4d8542c333b9b69035, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.869004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2a838-0f6f-4345-8ae7-20ea5de21d6d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.875959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b0bd3-a227-4936-a8b6-157e2f2d3d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17dbc4f2888543f2adb7eb6f9ece44b7, user_id=9bf127c12f0348f582ba984ebc581fa8, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:43.879295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b0bd3-a227-4936-a8b6-157e2f2d3d08 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:43.951797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 815/1632] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:43 2022] PUT /identity/v3/projects/6e1abf183aaf46f9beae2566b219e861/users/02b12fbbded94f4d8542c333b9b69035/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:43.952751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a838-0f6f-4345-8ae7-20ea5de21d6d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:43.966037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 818/1633] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:43 2022] PUT /identity/v3/projects/17dbc4f2888543f2adb7eb6f9ece44b7/users/9bf127c12f0348f582ba984ebc581fa8/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:43.966754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b0bd3-a227-4936-a8b6-157e2f2d3d08 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:44.086236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b0bd3-a227-4936-a8b6-157e2f2d3d08 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:44.096167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a838-0f6f-4345-8ae7-20ea5de21d6d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:44.107734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453ec42-0422-4dcc-a12f-38794ca726f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:44.108549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453ec42-0422-4dcc-a12f-38794ca726f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.108899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453ec42-0422-4dcc-a12f-38794ca726f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.109240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453ec42-0422-4dcc-a12f-38794ca726f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.113609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453ec42-0422-4dcc-a12f-38794ca726f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:44.116588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453ec42-0422-4dcc-a12f-38794ca726f1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:44.132572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 819/1634] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:44.134643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453ec42-0422-4dcc-a12f-38794ca726f1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:44.151794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453ec42-0422-4dcc-a12f-38794ca726f1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:44.156145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1efeda-d55d-4af5-9ee6-09b0458b5cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:44.156882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efeda-d55d-4af5-9ee6-09b0458b5cea admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.157250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efeda-d55d-4af5-9ee6-09b0458b5cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.157660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efeda-d55d-4af5-9ee6-09b0458b5cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.158602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1efeda-d55d-4af5-9ee6-09b0458b5cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:44.161082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1efeda-d55d-4af5-9ee6-09b0458b5cea admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:44.173074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 820/1635] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:44.174680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab8b2b-0952-4e19-b09a-844746f98576 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.175290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab8b2b-0952-4e19-b09a-844746f98576 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.175932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab8b2b-0952-4e19-b09a-844746f98576 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.178642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 821/1636] 10.222.0.30 () {54 vars in 912 bytes} [Sun Aug 7 12:21:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:21:44.183880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab8b2b-0952-4e19-b09a-844746f98576 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:44.197417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab8b2b-0952-4e19-b09a-844746f98576 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:44.199299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c094000-0c6d-489d-b5db-856746e55a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:44.199909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094000-0c6d-489d-b5db-856746e55a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.200190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094000-0c6d-489d-b5db-856746e55a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.200458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094000-0c6d-489d-b5db-856746e55a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.203650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa695c4-bd3f-41b0-952b-ddfb1441b917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:44.204414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa695c4-bd3f-41b0-952b-ddfb1441b917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.204648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa695c4-bd3f-41b0-952b-ddfb1441b917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.204928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa695c4-bd3f-41b0-952b-ddfb1441b917 admin admin] PATH_INFO: `/v3/projects/6e1abf183aaf46f9beae2566b219e861/users/02b12fbbded94f4d8542c333b9b69035/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.208031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa695c4-bd3f-41b0-952b-ddfb1441b917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1abf183aaf46f9beae2566b219e861, user_id=02b12fbbded94f4d8542c333b9b69035, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:44.212943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa695c4-bd3f-41b0-952b-ddfb1441b917 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:44.227183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 822/1637] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:44 2022] PUT /identity/v3/projects/6e1abf183aaf46f9beae2566b219e861/users/02b12fbbded94f4d8542c333b9b69035/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:44.228019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa695c4-bd3f-41b0-952b-ddfb1441b917 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:44.235529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:44.273302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c094000-0c6d-489d-b5db-856746e55a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:44.274955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c094000-0c6d-489d-b5db-856746e55a5a service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:44.287869 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 816/1638] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:44.289183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c289a-cc0a-46ae-b778-88af534d361c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.289466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c289a-cc0a-46ae-b778-88af534d361c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.289737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c289a-cc0a-46ae-b778-88af534d361c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.311652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-695c289a-cc0a-46ae-b778-88af534d361c None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:44.313380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-695c289a-cc0a-46ae-b778-88af534d361c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:44.345879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa695c4-bd3f-41b0-952b-ddfb1441b917 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:44.352245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a46935-53ad-4905-bb99-8403843a99bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:44.352919 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46935-53ad-4905-bb99-8403843a99bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.353206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46935-53ad-4905-bb99-8403843a99bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.353484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46935-53ad-4905-bb99-8403843a99bf admin admin] PATH_INFO: `/v3/projects/17dbc4f2888543f2adb7eb6f9ece44b7/users/9bf127c12f0348f582ba984ebc581fa8/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.355758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a46935-53ad-4905-bb99-8403843a99bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=17dbc4f2888543f2adb7eb6f9ece44b7, user_id=9bf127c12f0348f582ba984ebc581fa8, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:44.358657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a46935-53ad-4905-bb99-8403843a99bf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:44.378241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 823/1639] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:44 2022] PUT /identity/v3/projects/17dbc4f2888543f2adb7eb6f9ece44b7/users/9bf127c12f0348f582ba984ebc581fa8/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:44.380142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b601-6b03-43a0-b167-de9af5f0ba0d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.380672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b601-6b03-43a0-b167-de9af5f0ba0d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.384522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b601-6b03-43a0-b167-de9af5f0ba0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.427289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:44.432263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-00b8b601-6b03-43a0-b167-de9af5f0ba0d None None] MFA Rules not processed for user `02b12fbbded94f4d8542c333b9b69035`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:44.433555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-00b8b601-6b03-43a0-b167-de9af5f0ba0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:44.452871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 817/1640] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:44.453993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21279989-cbb2-481a-bb64-5ac24ae2bc12 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.454276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21279989-cbb2-481a-bb64-5ac24ae2bc12 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.454548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21279989-cbb2-481a-bb64-5ac24ae2bc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.496258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-21279989-cbb2-481a-bb64-5ac24ae2bc12 None None] MFA Rules not processed for user `9bf127c12f0348f582ba984ebc581fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:44.497489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-21279989-cbb2-481a-bb64-5ac24ae2bc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:44.531635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 824/1641] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:44.532593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b601-6b03-43a0-b167-de9af5f0ba0d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:44.587606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 818/1642] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:44.592970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21279989-cbb2-481a-bb64-5ac24ae2bc12 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:44.605512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b601-6b03-43a0-b167-de9af5f0ba0d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:44.610059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e65204-d409-45bc-8a87-21fcb419925c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:44.610815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e65204-d409-45bc-8a87-21fcb419925c service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.611276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e65204-d409-45bc-8a87-21fcb419925c service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.611667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e65204-d409-45bc-8a87-21fcb419925c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.654714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:44.669067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21279989-cbb2-481a-bb64-5ac24ae2bc12 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:44.682380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177ad6f-991b-45a7-8031-dc97346a34a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:44.683106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177ad6f-991b-45a7-8031-dc97346a34a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:44.683391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177ad6f-991b-45a7-8031-dc97346a34a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:44.683670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177ad6f-991b-45a7-8031-dc97346a34a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:44.713020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e65204-d409-45bc-8a87-21fcb419925c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:44.714557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e65204-d409-45bc-8a87-21fcb419925c service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:44.741699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 825/1643] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:44.784285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177ad6f-991b-45a7-8031-dc97346a34a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:44.786164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177ad6f-991b-45a7-8031-dc97346a34a8 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:44.815868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 819/1644] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:45.620068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e65204-d409-45bc-8a87-21fcb419925c service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:45.631733 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e65204-d409-45bc-8a87-21fcb419925c service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:45.639316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72026465-0666-480a-b7ca-c709017e604a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:45.640555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72026465-0666-480a-b7ca-c709017e604a service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:45.640555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72026465-0666-480a-b7ca-c709017e604a service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:45.640847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72026465-0666-480a-b7ca-c709017e604a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:45.652710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72026465-0666-480a-b7ca-c709017e604a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:45.654440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72026465-0666-480a-b7ca-c709017e604a service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:45.668114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 826/1645] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:45.786554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177ad6f-991b-45a7-8031-dc97346a34a8 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:45.798718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177ad6f-991b-45a7-8031-dc97346a34a8 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:45.804119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0bb95-1cba-441b-b3b3-2f3edba95fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:45.805384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0bb95-1cba-441b-b3b3-2f3edba95fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:45.805832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0bb95-1cba-441b-b3b3-2f3edba95fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:45.806306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0bb95-1cba-441b-b3b3-2f3edba95fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:45.819116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0bb95-1cba-441b-b3b3-2f3edba95fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:45.821985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0bb95-1cba-441b-b3b3-2f3edba95fa3 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:45.840027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 820/1646] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:45 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:46.003742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72026465-0666-480a-b7ca-c709017e604a service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:46.020737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72026465-0666-480a-b7ca-c709017e604a service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:46.031633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9f895-1d34-484b-9446-c29bbb803187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:46.033591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9f895-1d34-484b-9446-c29bbb803187 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:46.034165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9f895-1d34-484b-9446-c29bbb803187 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:46.034673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9f895-1d34-484b-9446-c29bbb803187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:46.143986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:46.366449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9f895-1d34-484b-9446-c29bbb803187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:46.366449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9f895-1d34-484b-9446-c29bbb803187 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:46.366449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 827/1647] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:46.458584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0bb95-1cba-441b-b3b3-2f3edba95fa3 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:46.637685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0bb95-1cba-441b-b3b3-2f3edba95fa3 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:46.651193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:46.651863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:46.652870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:46.655945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] PATH_INFO: `/v3/users/461eb7f2caaf4854b142ff70666a82a9` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:46.658269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=461eb7f2caaf4854b142ff70666a82a9)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:46.661827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:46.736715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbedb6-a043-4acb-975d-2248eda2b56a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:46.737398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbedb6-a043-4acb-975d-2248eda2b56a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:46.737840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbedb6-a043-4acb-975d-2248eda2b56a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:46.740194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 828/1648] 10.222.0.30 () {54 vars in 962 bytes} [Sun Aug 7 12:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:21:46.769262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c232-67af-4819-8550-c81d6d831899 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:46.775098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c232-67af-4819-8550-c81d6d831899 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:46.775818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c232-67af-4819-8550-c81d6d831899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:46.809151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3ba9c232-67af-4819-8550-c81d6d831899 None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:46.810638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3ba9c232-67af-4819-8550-c81d6d831899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:46.822715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 829/1649] 10.222.0.30 () {58 vars in 1059 bytes} [Sun Aug 7 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:46.828677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2106-89b6-4d9c-903b-ecdd7eadcc25 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:46.829174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2106-89b6-4d9c-903b-ecdd7eadcc25 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:46.829662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2106-89b6-4d9c-903b-ecdd7eadcc25 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:46.832488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 830/1650] 10.222.0.30 () {54 vars in 912 bytes} [Sun Aug 7 12:21:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 12:21:46.839416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a22863-a902-4758-8c20-83e4af2f3593 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:46.839957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a22863-a902-4758-8c20-83e4af2f3593 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:46.840395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a22863-a902-4758-8c20-83e4af2f3593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:46.863363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f0a22863-a902-4758-8c20-83e4af2f3593 None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:46.864719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f0a22863-a902-4758-8c20-83e4af2f3593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:46.878914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 831/1651] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:21:46.895782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a22863-a902-4758-8c20-83e4af2f3593 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:46.906485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a22863-a902-4758-8c20-83e4af2f3593 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:46.924881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d0447-c173-4644-82fd-733a373fe381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:46.925816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d0447-c173-4644-82fd-733a373fe381 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:46.926225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d0447-c173-4644-82fd-733a373fe381 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:46.927076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d0447-c173-4644-82fd-733a373fe381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:46.944299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d0447-c173-4644-82fd-733a373fe381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:46.945890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d0447-c173-4644-82fd-733a373fe381 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:46.960890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461eb7f2caaf4854b142ff70666a82a9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:46.965601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 832/1652] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:21:47.006526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461eb7f2caaf4854b142ff70666a82a9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:47.028799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461eb7f2caaf4854b142ff70666a82a9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:47.030499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461eb7f2caaf4854b142ff70666a82a9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:47.031625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461eb7f2caaf4854b142ff70666a82a9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:47.036864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:47.149256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 821/1653] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:21:46 2022] DELETE /identity/v3/users/461eb7f2caaf4854b142ff70666a82a9 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:47.155606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c21358-be89-4887-ad77-0c0d25a5a75e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:47.156149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c21358-be89-4887-ad77-0c0d25a5a75e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:47.156149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c21358-be89-4887-ad77-0c0d25a5a75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:47.206546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d6c21358-be89-4887-ad77-0c0d25a5a75e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:47.210131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6c21358-be89-4887-ad77-0c0d25a5a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:47.316729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 833/1654] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:47.328417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:47.331202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:47.472855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00429-71e3-466a-a64e-c188e1ec6186 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:47.483418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d019f2-6fc7-44ae-bb5e-539133798a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:47.491452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d019f2-6fc7-44ae-bb5e-539133798a46 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:47.491452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d019f2-6fc7-44ae-bb5e-539133798a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:47.491452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d019f2-6fc7-44ae-bb5e-539133798a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:47.501905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d019f2-6fc7-44ae-bb5e-539133798a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:47.504829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d019f2-6fc7-44ae-bb5e-539133798a46 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:47.518592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 822/1655] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:21:49.142008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c21358-be89-4887-ad77-0c0d25a5a75e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:49.143531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6c21358-be89-4887-ad77-0c0d25a5a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:49.167770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c21358-be89-4887-ad77-0c0d25a5a75e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:49.179341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:49.181027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:49.181590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:49.182012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] PATH_INFO: `/v3/projects/5c15466dfd944d929c9e5db6f9f144bb` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:49.183532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c15466dfd944d929c9e5db6f9f144bb)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:49.187195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:49.380528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c15466dfd944d929c9e5db6f9f144bb', 'request_id': 'req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:49.455951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c15466dfd944d929c9e5db6f9f144bb', 'request_id': 'req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:49.475144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c15466dfd944d929c9e5db6f9f144bb', 'request_id': 'req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:49.520101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] The token cache is being invalidate because project 5c15466dfd944d929c9e5db6f9f144bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:21:49.521305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:49.523275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:49.528831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 834/1656] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:21:49 2022] DELETE /identity/v3/projects/5c15466dfd944d929c9e5db6f9f144bb => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:49.530795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d019f2-6fc7-44ae-bb5e-539133798a46 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:49.564957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d019f2-6fc7-44ae-bb5e-539133798a46 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:49.792000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:49.792843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:49.793132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:49.793415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] PATH_INFO: `/v3/users/9bf127c12f0348f582ba984ebc581fa8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:49.795779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf127c12f0348f582ba984ebc581fa8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:49.799781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:49.807770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:49.807770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:49.838130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567a40a-ec43-4bd3-9cdb-c3e195044a6b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:49.863591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:49.864801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:49.865396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:49.866012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] PATH_INFO: `/v3/users/f68c2c20add245868f00e405885fac16` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:49.867578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f68c2c20add245868f00e405885fac16)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:49.868779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:21:49.868779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Aug 07 12:21:49.868779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: Aug 07 12:21:49.872304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:50.110154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf127c12f0348f582ba984ebc581fa8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.137416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:50.141281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf127c12f0348f582ba984ebc581fa8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.160391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf127c12f0348f582ba984ebc581fa8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.160391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf127c12f0348f582ba984ebc581fa8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.160391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fcb2aa8f-8e39-40a7-8bde-4171867f7b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf127c12f0348f582ba984ebc581fa8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.188750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f68c2c20add245868f00e405885fac16'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.225735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f68c2c20add245868f00e405885fac16'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.238517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 823/1657] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:21:49 2022] DELETE /identity/v3/users/9bf127c12f0348f582ba984ebc581fa8 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:50.249368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f68c2c20add245868f00e405885fac16'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.250392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f68c2c20add245868f00e405885fac16'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.267044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f68c2c20add245868f00e405885fac16'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:50.324654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 835/1658] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:21:49 2022] DELETE /identity/v3/users/f68c2c20add245868f00e405885fac16 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:50.326866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0cd1-a4e6-4cf7-87fb-f512cb39fd37 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:50.327414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0cd1-a4e6-4cf7-87fb-f512cb39fd37 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:50.327829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0cd1-a4e6-4cf7-87fb-f512cb39fd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:50.365733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-5d1e0cd1-a4e6-4cf7-87fb-f512cb39fd37 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:50.366986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5d1e0cd1-a4e6-4cf7-87fb-f512cb39fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:50.391064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:50.484059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 824/1659] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:50.507974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:50.509672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:50.641033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52f742-0131-424d-9750-09d4bcc38b61 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:50.644909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:50.645529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:50.645804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:50.646076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:50.680982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:50.680982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:50.694607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 836/1660] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:51.830265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e0cd1-a4e6-4cf7-87fb-f512cb39fd37 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:51.831798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5d1e0cd1-a4e6-4cf7-87fb-f512cb39fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:51.845986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e0cd1-a4e6-4cf7-87fb-f512cb39fd37 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:51.852864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:51.853973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:51.854396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:51.854803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] PATH_INFO: `/v3/projects/4463509e9402446f93dd4334865e5b36` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:51.862922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4463509e9402446f93dd4334865e5b36)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:51.868623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: WARNING py.warnings [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 12:21:51.868623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: warnings.warn(msg) Aug 07 12:21:51.868623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: Aug 07 12:21:51.870179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:52.089379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4463509e9402446f93dd4334865e5b36', 'request_id': 'req-cf387808-dd8f-43c2-96d9-2fe7b07906aa'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.187457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4463509e9402446f93dd4334865e5b36', 'request_id': 'req-cf387808-dd8f-43c2-96d9-2fe7b07906aa'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.198616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4463509e9402446f93dd4334865e5b36', 'request_id': 'req-cf387808-dd8f-43c2-96d9-2fe7b07906aa'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.258525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] The token cache is being invalidate because project 4463509e9402446f93dd4334865e5b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:21:52.262165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.263100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cf387808-dd8f-43c2-96d9-2fe7b07906aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.271138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 825/1661] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:21:51 2022] DELETE /identity/v3/projects/4463509e9402446f93dd4334865e5b36 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:52.271876 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:52.272346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:52.273498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997f6c8-eea5-4e9b-89b7-ab4d44b6ef51 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:52.273786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997f6c8-eea5-4e9b-89b7-ab4d44b6ef51 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:52.276261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997f6c8-eea5-4e9b-89b7-ab4d44b6ef51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:52.296647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcde99-883b-4b0c-8841-0c7413a61d70 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:52.322984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7997f6c8-eea5-4e9b-89b7-ab4d44b6ef51 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:52.324242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7997f6c8-eea5-4e9b-89b7-ab4d44b6ef51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:52.490714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:52.491685 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:52.492209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:52.498285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] PATH_INFO: `/v3/projects/17dbc4f2888543f2adb7eb6f9ece44b7` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:52.498285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] RBAC: Authorizing `identity:delete_project(project_id=17dbc4f2888543f2adb7eb6f9ece44b7)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:52.499584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:52.511701 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 826/1662] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:52.519414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa254-91bc-4d5f-a532-6d1158efe17f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:52.521072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa254-91bc-4d5f-a532-6d1158efe17f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:52.521072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa254-91bc-4d5f-a532-6d1158efe17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:52.565769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-becaa254-91bc-4d5f-a532-6d1158efe17f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:52.567336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-becaa254-91bc-4d5f-a532-6d1158efe17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:52.588146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 827/1663] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:52.592262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaa254-91bc-4d5f-a532-6d1158efe17f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:52.610480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaa254-91bc-4d5f-a532-6d1158efe17f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:52.626020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:52.627112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:52.627561 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:52.628051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:52.628848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:52.630514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:52.636832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:52.680450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 828/1664] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:21:52.684879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:52.685697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17dbc4f2888543f2adb7eb6f9ece44b7', 'request_id': 'req-ae1fbd6e-4082-42a0-8087-06c03f5b087d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.697736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:52.721253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbdd7bb-2889-423b-820f-e1613fa38711 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:52.774537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:52.787575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17dbc4f2888543f2adb7eb6f9ece44b7', 'request_id': 'req-ae1fbd6e-4082-42a0-8087-06c03f5b087d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.795702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17dbc4f2888543f2adb7eb6f9ece44b7', 'request_id': 'req-ae1fbd6e-4082-42a0-8087-06c03f5b087d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.822268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] The token cache is being invalidate because project 17dbc4f2888543f2adb7eb6f9ece44b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:21:52.824375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.826630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ae1fbd6e-4082-42a0-8087-06c03f5b087d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:52.830578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 837/1665] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:21:52 2022] DELETE /identity/v3/projects/17dbc4f2888543f2adb7eb6f9ece44b7 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:52.847838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8454ff-e3cf-4c21-9a56-b9f5089ee1e8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:52.853770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8454ff-e3cf-4c21-9a56-b9f5089ee1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:52.853770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8454ff-e3cf-4c21-9a56-b9f5089ee1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:52.894075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:52.894790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:52.895544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:52.895896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:52.897288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2f8454ff-e3cf-4c21-9a56-b9f5089ee1e8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:52.898117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:52.900467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2f8454ff-e3cf-4c21-9a56-b9f5089ee1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:52.902029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:52.933741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 838/1666] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:52.935192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15b30-a1e7-47d2-84f8-ae8644082223 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:52.935483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15b30-a1e7-47d2-84f8-ae8644082223 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:52.935760 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15b30-a1e7-47d2-84f8-ae8644082223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:52.966022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a1d15b30-a1e7-47d2-84f8-ae8644082223 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:52.968097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a1d15b30-a1e7-47d2-84f8-ae8644082223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:53.000857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 839/1667] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:53.014364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d15b30-a1e7-47d2-84f8-ae8644082223 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:53.070929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d15b30-a1e7-47d2-84f8-ae8644082223 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:53.073677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 829/1668] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:52 2022] POST /identity/v3/projects => generated 371 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:53.077653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:53.079683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:53.253014 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:53.253644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:53.253955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:53.254204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:53.257692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:21:53.258004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:53.259913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:53.275958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66fffe-9818-45c0-bef3-177ceef97db3 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:53.285336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 840/1669] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:21:53.289825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:53.290694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:53.291199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:53.291715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:53.292551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:53.295696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:53.296258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:53.322921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049b825-b6b0-49f2-a632-1e09f2d8e853 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:53.337174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b29c7-5c83-4c12-9ee2-2484ad1504d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:53.339436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b29c7-5c83-4c12-9ee2-2484ad1504d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:53.340672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b29c7-5c83-4c12-9ee2-2484ad1504d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:53.341188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b29c7-5c83-4c12-9ee2-2484ad1504d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:53.342080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b29c7-5c83-4c12-9ee2-2484ad1504d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:53.345234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b29c7-5c83-4c12-9ee2-2484ad1504d8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:53.405841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 830/1670] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:53 2022] POST /identity/v3/users => generated 286 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:53.407071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:53.411210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:53.444983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896accd-7857-497c-abac-5dcc49840aa2 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:53.459286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 841/1671] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:53 2022] POST /identity/v3/projects => generated 369 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:53.460931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b29c7-5c83-4c12-9ee2-2484ad1504d8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:53.473516 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:53.474180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:53.474963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:53.480919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] PATH_INFO: `/v3/users/3d1b8b586b4949109e468eb3c744b63c` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:53.485245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d1b8b586b4949109e468eb3c744b63c)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:53.491279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:53.665434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b29c7-5c83-4c12-9ee2-2484ad1504d8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:53.676331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:53.676993 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:53.677280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:53.677656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:53.678308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:53.686084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:53.715988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 842/1672] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:53.717028 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:53.720582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:53.754614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e1f3c-bc50-4b7c-b405-586b800d40e0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:53.762260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:53.788154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:53.788822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:53.789242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] PATH_INFO: `/v3/users/9e9d3a02e1c8420ca1993eedc5009601` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:53.819837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9d3a02e1c8420ca1993eedc5009601)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:53.835839 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d1b8b586b4949109e468eb3c744b63c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:53.836222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:53.868079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d1b8b586b4949109e468eb3c744b63c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:53.884226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d1b8b586b4949109e468eb3c744b63c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:53.886451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d1b8b586b4949109e468eb3c744b63c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:53.886556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d1b8b586b4949109e468eb3c744b63c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:53.974820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 831/1673] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:21:53 2022] DELETE /identity/v3/users/3d1b8b586b4949109e468eb3c744b63c => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:53.975773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:53.978856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:54.116850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dc9f3-ffa1-4705-9246-64af33d5d5dc admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:54.127731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:54.128527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.128815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.129266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.130068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:54.132275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:54.183604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9d3a02e1c8420ca1993eedc5009601'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:54.208099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9d3a02e1c8420ca1993eedc5009601'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:54.242675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9d3a02e1c8420ca1993eedc5009601'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:54.243374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 832/1674] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:53 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:54.245221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:54.245570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9d3a02e1c8420ca1993eedc5009601'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:54.246683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f080702f-2f37-4e60-859f-265613cc8f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9d3a02e1c8420ca1993eedc5009601'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:54.248422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:54.284495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434946-5689-4d85-a144-b9ccc09e7958 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:54.295780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426f0c6-b7ef-451d-bbe1-dbf4f7466e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:54.296669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426f0c6-b7ef-451d-bbe1-dbf4f7466e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.297083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426f0c6-b7ef-451d-bbe1-dbf4f7466e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.297503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426f0c6-b7ef-451d-bbe1-dbf4f7466e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.298309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426f0c6-b7ef-451d-bbe1-dbf4f7466e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:54.300822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426f0c6-b7ef-451d-bbe1-dbf4f7466e54 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:54.331981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 833/1675] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:54.335427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb721a-41fe-4c21-94cc-13e59fa67702 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.336838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb721a-41fe-4c21-94cc-13e59fa67702 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.342720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb721a-41fe-4c21-94cc-13e59fa67702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.371366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 843/1676] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:21:53 2022] DELETE /identity/v3/users/9e9d3a02e1c8420ca1993eedc5009601 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:54.374042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497f530-ca6e-4ed3-ab59-d1262e2bdf6b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.374617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497f530-ca6e-4ed3-ab59-d1262e2bdf6b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.375175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497f530-ca6e-4ed3-ab59-d1262e2bdf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.418483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5497f530-ca6e-4ed3-ab59-d1262e2bdf6b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:54.419884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5497f530-ca6e-4ed3-ab59-d1262e2bdf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:54.423557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e4eb721a-41fe-4c21-94cc-13e59fa67702 None None] MFA Rules not processed for user `5401baffbd794016b01d109e8e2529ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:54.424518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e4eb721a-41fe-4c21-94cc-13e59fa67702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:54.567029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 844/1677] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:54.569519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497f530-ca6e-4ed3-ab59-d1262e2bdf6b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:54.585231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5497f530-ca6e-4ed3-ab59-d1262e2bdf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:54.607029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 834/1678] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:54.607029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb721a-41fe-4c21-94cc-13e59fa67702 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:54.613770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497f530-ca6e-4ed3-ab59-d1262e2bdf6b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:54.622851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8906bd-06e4-457c-b2b9-a059d8fcf762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:54.623704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb721a-41fe-4c21-94cc-13e59fa67702 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:54.624580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8906bd-06e4-457c-b2b9-a059d8fcf762 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.625030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8906bd-06e4-457c-b2b9-a059d8fcf762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.625957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8906bd-06e4-457c-b2b9-a059d8fcf762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.627514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8906bd-06e4-457c-b2b9-a059d8fcf762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:54.628792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb061b16-ce73-4210-b152-49048c76b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:54.629555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb061b16-ce73-4210-b152-49048c76b1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.630102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb061b16-ce73-4210-b152-49048c76b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.630461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb061b16-ce73-4210-b152-49048c76b1b1 admin admin] PATH_INFO: `/v3/projects/40bb9b119a7744f9a96ea8453e0850ac/users/d2989133ec404c3ea79ce4efcaf5b9bd/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.635745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8906bd-06e4-457c-b2b9-a059d8fcf762 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:54.657453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 845/1679] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:54.660601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787e2a-4725-4994-9580-d2d74331d679 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.664433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb061b16-ce73-4210-b152-49048c76b1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40bb9b119a7744f9a96ea8453e0850ac, user_id=d2989133ec404c3ea79ce4efcaf5b9bd, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:54.664570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787e2a-4725-4994-9580-d2d74331d679 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.664923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787e2a-4725-4994-9580-d2d74331d679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.667782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb061b16-ce73-4210-b152-49048c76b1b1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:54.695998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-95787e2a-4725-4994-9580-d2d74331d679 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:54.697649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-95787e2a-4725-4994-9580-d2d74331d679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:54.710631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 846/1680] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:54.711847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787e2a-4725-4994-9580-d2d74331d679 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:54.713255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-95787e2a-4725-4994-9580-d2d74331d679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:54.773341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 835/1681] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:54 2022] PUT /identity/v3/projects/40bb9b119a7744f9a96ea8453e0850ac/users/d2989133ec404c3ea79ce4efcaf5b9bd/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:54.774651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5a10-e77f-4d98-a721-016ca08a0305 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.775111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5a10-e77f-4d98-a721-016ca08a0305 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.780148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5a10-e77f-4d98-a721-016ca08a0305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.813462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-375f5a10-e77f-4d98-a721-016ca08a0305 None None] MFA Rules not processed for user `5401baffbd794016b01d109e8e2529ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:54.819412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-375f5a10-e77f-4d98-a721-016ca08a0305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:54.819549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787e2a-4725-4994-9580-d2d74331d679 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:54.885086 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76020ac4-71b2-4662-b5c7-9d681b104761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:54.886851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020ac4-71b2-4662-b5c7-9d681b104761 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:54.887274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020ac4-71b2-4662-b5c7-9d681b104761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:54.888969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020ac4-71b2-4662-b5c7-9d681b104761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:54.988813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 836/1682] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:54.990413 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f5a10-e77f-4d98-a721-016ca08a0305 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:55.080092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76020ac4-71b2-4662-b5c7-9d681b104761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:55.091712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76020ac4-71b2-4662-b5c7-9d681b104761 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:55.115097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 847/1683] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:55.116639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76020ac4-71b2-4662-b5c7-9d681b104761 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:55.136502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76020ac4-71b2-4662-b5c7-9d681b104761 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:55.154358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:55.160328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14480716-6fed-4b4c-9cff-af1615f54c68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:55.167434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480716-6fed-4b4c-9cff-af1615f54c68 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:55.167434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480716-6fed-4b4c-9cff-af1615f54c68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:55.167434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480716-6fed-4b4c-9cff-af1615f54c68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:55.172036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f5a10-e77f-4d98-a721-016ca08a0305 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:55.184737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:55.184737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:55.184737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:55.184737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:55.185580 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:55.188031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:55.195977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14480716-6fed-4b4c-9cff-af1615f54c68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:55.198381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14480716-6fed-4b4c-9cff-af1615f54c68 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:55.215896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 848/1684] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:55.217344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456df452-2054-418a-b78c-0efbea831f89 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:55.217639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456df452-2054-418a-b78c-0efbea831f89 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:55.217920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456df452-2054-418a-b78c-0efbea831f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:55.219593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 837/1685] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:55.222291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:55.225951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:55.333444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-456df452-2054-418a-b78c-0efbea831f89 None None] MFA Rules not processed for user `d2989133ec404c3ea79ce4efcaf5b9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:55.339312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-456df452-2054-418a-b78c-0efbea831f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:55.358971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:55.497988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fd0e1-9ece-4472-a777-b79f7fe683bb admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:55.514648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f301d93-6388-43b8-92ba-b2520a0bea5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:55.515497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f301d93-6388-43b8-92ba-b2520a0bea5a service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:55.515856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f301d93-6388-43b8-92ba-b2520a0bea5a service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:55.516344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f301d93-6388-43b8-92ba-b2520a0bea5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:55.536202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f301d93-6388-43b8-92ba-b2520a0bea5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:55.538266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f301d93-6388-43b8-92ba-b2520a0bea5a service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:55.564322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 849/1686] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:55.565301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456df452-2054-418a-b78c-0efbea831f89 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:55.566079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 838/1687] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:55.568812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f301d93-6388-43b8-92ba-b2520a0bea5a service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:55.593243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456df452-2054-418a-b78c-0efbea831f89 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:55.594978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f301d93-6388-43b8-92ba-b2520a0bea5a service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:55.603894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a0210-8501-44ec-b94f-3a378eb61056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:55.604589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a0210-8501-44ec-b94f-3a378eb61056 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:55.604864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a0210-8501-44ec-b94f-3a378eb61056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:55.609381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a0210-8501-44ec-b94f-3a378eb61056 admin admin] PATH_INFO: `/v3/projects/b004823ea9d24e7ab4f5f94860fdd04a/users/e2bfd7b11a1349da9edb69744bcb4279/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:55.609515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a185486-5e8c-40fd-82f1-1999320e9ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:55.609515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a185486-5e8c-40fd-82f1-1999320e9ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:55.609515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a185486-5e8c-40fd-82f1-1999320e9ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:55.609515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a185486-5e8c-40fd-82f1-1999320e9ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:55.609787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a185486-5e8c-40fd-82f1-1999320e9ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:55.610432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a185486-5e8c-40fd-82f1-1999320e9ef4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:55.658135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a0210-8501-44ec-b94f-3a378eb61056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b004823ea9d24e7ab4f5f94860fdd04a, user_id=e2bfd7b11a1349da9edb69744bcb4279, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:55.662402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a0210-8501-44ec-b94f-3a378eb61056 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:55.784646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 850/1688] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:55 2022] PUT /identity/v3/projects/b004823ea9d24e7ab4f5f94860fdd04a/users/e2bfd7b11a1349da9edb69744bcb4279/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:55.784818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 839/1689] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:55 2022] POST /identity/v3/projects => generated 371 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:55.790430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a0210-8501-44ec-b94f-3a378eb61056 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:55.790822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6bd35-1be2-439b-ae91-a517fea0bf9e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:55.790822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6bd35-1be2-439b-ae91-a517fea0bf9e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:55.790822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6bd35-1be2-439b-ae91-a517fea0bf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:55.872035 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-79c6bd35-1be2-439b-ae91-a517fea0bf9e None None] MFA Rules not processed for user `e2bfd7b11a1349da9edb69744bcb4279`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:55.879891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-79c6bd35-1be2-439b-ae91-a517fea0bf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:55.948870 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a0210-8501-44ec-b94f-3a378eb61056 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.064298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d63af-e4ce-46f2-a261-9ebd6e9f36d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.064936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d63af-e4ce-46f2-a261-9ebd6e9f36d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.065218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d63af-e4ce-46f2-a261-9ebd6e9f36d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.065532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d63af-e4ce-46f2-a261-9ebd6e9f36d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.066742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d63af-e4ce-46f2-a261-9ebd6e9f36d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:56.068933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d63af-e4ce-46f2-a261-9ebd6e9f36d6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:56.073742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 840/1690] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:21:56.078691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6bd35-1be2-439b-ae91-a517fea0bf9e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:56.104093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6bd35-1be2-439b-ae91-a517fea0bf9e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.122380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3147c0-a49b-4daa-9fd6-d43f90514879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.123303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3147c0-a49b-4daa-9fd6-d43f90514879 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.123718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3147c0-a49b-4daa-9fd6-d43f90514879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.124318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3147c0-a49b-4daa-9fd6-d43f90514879 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.125089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3147c0-a49b-4daa-9fd6-d43f90514879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:56.127205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3147c0-a49b-4daa-9fd6-d43f90514879 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:56.209598 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 851/1691] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:55 2022] POST /identity/v3/users => generated 292 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:56.212858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d63af-e4ce-46f2-a261-9ebd6e9f36d6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:56.224554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d63af-e4ce-46f2-a261-9ebd6e9f36d6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.230743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a7141-830e-441a-a036-6b3fdd77a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.231555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a7141-830e-441a-a036-6b3fdd77a97f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.231927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a7141-830e-441a-a036-6b3fdd77a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.232309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a7141-830e-441a-a036-6b3fdd77a97f admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.233072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a7141-830e-441a-a036-6b3fdd77a97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:56.234884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a7141-830e-441a-a036-6b3fdd77a97f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:56.243182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 841/1692] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:21:56 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:56.247475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3147c0-a49b-4daa-9fd6-d43f90514879 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:56.266772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 852/1693] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:56.268292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a7141-830e-441a-a036-6b3fdd77a97f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:56.298115 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a7141-830e-441a-a036-6b3fdd77a97f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.428547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3147c0-a49b-4daa-9fd6-d43f90514879 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.445478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea34eea-b5ba-45ec-965e-cccc1c414f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.450565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea34eea-b5ba-45ec-965e-cccc1c414f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.451167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea34eea-b5ba-45ec-965e-cccc1c414f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.451642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea34eea-b5ba-45ec-965e-cccc1c414f80 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.452539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea34eea-b5ba-45ec-965e-cccc1c414f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:56.455982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea34eea-b5ba-45ec-965e-cccc1c414f80 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:56.483921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8befa-bc96-4f01-9d35-13d4f4e1eea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.487202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8befa-bc96-4f01-9d35-13d4f4e1eea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.487202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8befa-bc96-4f01-9d35-13d4f4e1eea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.487202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8befa-bc96-4f01-9d35-13d4f4e1eea7 admin admin] PATH_INFO: `/v3/projects/d1da10c585b5418382f58dab93b46365/users/99470ebdf01a444691a1b0422f00c644/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.525738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8befa-bc96-4f01-9d35-13d4f4e1eea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1da10c585b5418382f58dab93b46365, user_id=99470ebdf01a444691a1b0422f00c644, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:56.536668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8befa-bc96-4f01-9d35-13d4f4e1eea7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:56.579793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 842/1694] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:21:56 2022] POST /identity/v3/users => generated 289 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:21:56.580634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea34eea-b5ba-45ec-965e-cccc1c414f80 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:56.607482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea34eea-b5ba-45ec-965e-cccc1c414f80 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.619748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580b73e-a1f3-4e74-afd1-cedb8bf2eb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.620459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580b73e-a1f3-4e74-afd1-cedb8bf2eb4a admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.620914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580b73e-a1f3-4e74-afd1-cedb8bf2eb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.621328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580b73e-a1f3-4e74-afd1-cedb8bf2eb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.622326 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580b73e-a1f3-4e74-afd1-cedb8bf2eb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:56.628867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580b73e-a1f3-4e74-afd1-cedb8bf2eb4a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:56.636655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 853/1695] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:56 2022] PUT /identity/v3/projects/d1da10c585b5418382f58dab93b46365/users/99470ebdf01a444691a1b0422f00c644/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:56.640696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419b43-e922-4212-a031-8c885004a98c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.641168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419b43-e922-4212-a031-8c885004a98c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.641769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419b43-e922-4212-a031-8c885004a98c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.644241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 854/1696] 10.222.0.30 () {54 vars in 912 bytes} [Sun Aug 7 12:21:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:21:56.648514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9419b43-e922-4212-a031-8c885004a98c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:56.650575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 843/1697] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:21:56.651665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478ef12-c0fa-4e22-92f0-12a4c0f7e795 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.651955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478ef12-c0fa-4e22-92f0-12a4c0f7e795 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.652232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478ef12-c0fa-4e22-92f0-12a4c0f7e795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.737695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d478ef12-c0fa-4e22-92f0-12a4c0f7e795 None None] MFA Rules not processed for user `f7aa31ea778140189b041939e3f01aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:56.739511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d478ef12-c0fa-4e22-92f0-12a4c0f7e795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:56.862507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9419b43-e922-4212-a031-8c885004a98c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.878162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.879114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.879405 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.879684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.885162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:56.893160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:56.906058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 844/1698] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:21:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:56.907047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478ef12-c0fa-4e22-92f0-12a4c0f7e795 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:56.923399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 855/1699] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:56.924234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:56.930396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:56.939702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478ef12-c0fa-4e22-92f0-12a4c0f7e795 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.950702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad912d-c415-41c4-8101-c6971d34e7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.951484 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad912d-c415-41c4-8101-c6971d34e7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.951766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad912d-c415-41c4-8101-c6971d34e7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.957595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad912d-c415-41c4-8101-c6971d34e7c6 admin admin] PATH_INFO: `/v3/projects/58ffbd1872564385b10c359289567ecc/users/9f03ab4adbba48eb9ac3d8540a7025b8/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.969013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed399e6-a514-49c6-86f3-72748b040ff7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:56.981768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:56.982368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:56.982649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:56.982928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] PATH_INFO: `/v3/projects/71ef7979b9fc4098a9219118616fb096` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:56.984579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ef7979b9fc4098a9219118616fb096)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:56.987265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:57.000273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad912d-c415-41c4-8101-c6971d34e7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ffbd1872564385b10c359289567ecc, user_id=9f03ab4adbba48eb9ac3d8540a7025b8, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:57.010582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad912d-c415-41c4-8101-c6971d34e7c6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:57.085845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 845/1700] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:56 2022] PUT /identity/v3/projects/58ffbd1872564385b10c359289567ecc/users/9f03ab4adbba48eb9ac3d8540a7025b8/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:57.086737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad912d-c415-41c4-8101-c6971d34e7c6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:57.148153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ef7979b9fc4098a9219118616fb096', 'request_id': 'req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:57.204385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad912d-c415-41c4-8101-c6971d34e7c6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:57.256920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ef7979b9fc4098a9219118616fb096', 'request_id': 'req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:57.266235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ef7979b9fc4098a9219118616fb096', 'request_id': 'req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:57.301333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] The token cache is being invalidate because project 71ef7979b9fc4098a9219118616fb096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:21:57.301333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:57.302308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:57.312020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 856/1701] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:21:56 2022] DELETE /identity/v3/projects/71ef7979b9fc4098a9219118616fb096 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:57.312020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:57.314379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:57.321326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:57.359825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:57.401600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:57.402357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:57.402644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:57.402954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] PATH_INFO: `/v3/projects/d1da10c585b5418382f58dab93b46365/users/99470ebdf01a444691a1b0422f00c644/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:57.409386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1da10c585b5418382f58dab93b46365, user_id=99470ebdf01a444691a1b0422f00c644, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:57.415504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:57.441208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 846/1702] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:57 2022] PUT /identity/v3/projects/d1da10c585b5418382f58dab93b46365/users/99470ebdf01a444691a1b0422f00c644/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:57.442088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:57.444409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:57.458108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bee721-b0cb-47ad-b118-b4b6fdd0ce42 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:57.600761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:57.601552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:57.601838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:57.602115 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:57.604326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:57.622922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684e7f4-3529-47d9-971d-3f1749a37f02 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:57.626696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:57.628317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:57.633099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6c4c0-5018-4af1-8640-8a82371a4ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:57.633999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c4c0-5018-4af1-8640-8a82371a4ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:57.634284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c4c0-5018-4af1-8640-8a82371a4ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:57.634562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c4c0-5018-4af1-8640-8a82371a4ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:57.635447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6c4c0-5018-4af1-8640-8a82371a4ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:57.637641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6c4c0-5018-4af1-8640-8a82371a4ef0 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:57.652825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 857/1703] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:57.653700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:57.655103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:57.658572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 847/1704] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:21:57.659960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe23f90-552e-4cfd-9ebc-a57553bd003e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:57.660247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe23f90-552e-4cfd-9ebc-a57553bd003e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:57.660525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe23f90-552e-4cfd-9ebc-a57553bd003e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:57.676125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7293927-480a-445e-b358-5416c03da180 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:57.684938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:57.685600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:57.685882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:57.686154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] PATH_INFO: `/v3/projects/d92481181f434ae49c232fd807f4b896` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:57.709283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92481181f434ae49c232fd807f4b896)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:57.712032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-cfe23f90-552e-4cfd-9ebc-a57553bd003e None None] MFA Rules not processed for user `90dc7b751dd14c6f80205e709826423b`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:57.713738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cfe23f90-552e-4cfd-9ebc-a57553bd003e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:57.714799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:57.828565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 848/1705] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:57.839814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe23f90-552e-4cfd-9ebc-a57553bd003e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:57.841797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cfe23f90-552e-4cfd-9ebc-a57553bd003e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:57.864037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe23f90-552e-4cfd-9ebc-a57553bd003e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:57.889654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:57.890711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:57.891342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:57.891821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] PATH_INFO: `/v3/users/02b12fbbded94f4d8542c333b9b69035` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:57.894389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b12fbbded94f4d8542c333b9b69035)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:57.896476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92481181f434ae49c232fd807f4b896', 'request_id': 'req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:57.899330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:57.953854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92481181f434ae49c232fd807f4b896', 'request_id': 'req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:57.967935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92481181f434ae49c232fd807f4b896', 'request_id': 'req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.002588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] The token cache is being invalidate because project d92481181f434ae49c232fd807f4b896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:21:58.004632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.010032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ad9248c4-7b9e-4f40-8ec9-d1665d25bd11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.012677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 858/1706] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:21:57 2022] DELETE /identity/v3/projects/d92481181f434ae49c232fd807f4b896 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:58.020063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260439b5-03c1-4dd6-b1eb-0bbeeee6e86c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:58.022277 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260439b5-03c1-4dd6-b1eb-0bbeeee6e86c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:58.022277 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260439b5-03c1-4dd6-b1eb-0bbeeee6e86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:58.028884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:58.072657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-260439b5-03c1-4dd6-b1eb-0bbeeee6e86c None None] MFA Rules not processed for user `99470ebdf01a444691a1b0422f00c644`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:58.074186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-260439b5-03c1-4dd6-b1eb-0bbeeee6e86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:58.124937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:58.151211 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b12fbbded94f4d8542c333b9b69035'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.204010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b12fbbded94f4d8542c333b9b69035'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.230314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b12fbbded94f4d8542c333b9b69035'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.232594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b12fbbded94f4d8542c333b9b69035'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.237433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b12fbbded94f4d8542c333b9b69035'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.256907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 859/1707] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:58.258565 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665e9ce-5992-4b83-ae14-393572b7e70a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:58.259239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665e9ce-5992-4b83-ae14-393572b7e70a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:58.260058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665e9ce-5992-4b83-ae14-393572b7e70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:58.308329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5665e9ce-5992-4b83-ae14-393572b7e70a None None] MFA Rules not processed for user `c8fb20cfc3b6479198059b4383f47527`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:58.310212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5665e9ce-5992-4b83-ae14-393572b7e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:58.355477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 849/1708] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:21:57 2022] DELETE /identity/v3/users/02b12fbbded94f4d8542c333b9b69035 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:58.374690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:58.378528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:58.433455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 860/1709] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:58.435768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d3c42-abd8-42f1-91b4-a03266637c27 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:58.436207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d3c42-abd8-42f1-91b4-a03266637c27 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:58.444071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d3c42-abd8-42f1-91b4-a03266637c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:58.462175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-078d3c42-abd8-42f1-91b4-a03266637c27 None None] MFA Rules not processed for user `90dc7b751dd14c6f80205e709826423b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:58.464134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-078d3c42-abd8-42f1-91b4-a03266637c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:58.497736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:58.549925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 861/1710] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:58.551212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d3c42-abd8-42f1-91b4-a03266637c27 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:58.553206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-078d3c42-abd8-42f1-91b4-a03266637c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:58.576076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d3c42-abd8-42f1-91b4-a03266637c27 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:58.576532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f111c87-fd15-4b01-8ecf-b149e6a11e46 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:58.584433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:58.585330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:58.585618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:58.585902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] PATH_INFO: `/v3/projects/58ffbd1872564385b10c359289567ecc/users/9f03ab4adbba48eb9ac3d8540a7025b8/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:58.586893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:58.587688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:58.587904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ffbd1872564385b10c359289567ecc, user_id=9f03ab4adbba48eb9ac3d8540a7025b8, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:58.588010 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:58.588228 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] PATH_INFO: `/v3/users/0c30712dfe124f1282befda8038f216a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:58.591046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:58.591369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c30712dfe124f1282befda8038f216a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:58.593421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:58.604478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 850/1711] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:21:58 2022] PUT /identity/v3/projects/58ffbd1872564385b10c359289567ecc/users/9f03ab4adbba48eb9ac3d8540a7025b8/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:58.606003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:58.614550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:58.645688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:58.783157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a36e9-8924-44d6-8240-fa727ed97795 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:58.791907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:58.792556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:58.792836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:58.793109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:58.795528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:58.879130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c30712dfe124f1282befda8038f216a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.896619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c30712dfe124f1282befda8038f216a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.926867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:58.929467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c30712dfe124f1282befda8038f216a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.932084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c30712dfe124f1282befda8038f216a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.932196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:58.932340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c30712dfe124f1282befda8038f216a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:58.957227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 851/1712] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:58.960565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:58.962535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:59.024181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 862/1713] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:21:58 2022] DELETE /identity/v3/users/0c30712dfe124f1282befda8038f216a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:59.025057 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:59.026421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:59.061328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21926a-b1c1-4cd7-9d13-d1d8ab50bce7 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:59.158233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74dc60-6340-4107-a57a-9b878e97e3b1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:59.161364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:59.162414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:59.162971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:59.163391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] PATH_INFO: `/v3/users/415a32d179144aeb97aff3ea81835835` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:59.165279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=415a32d179144aeb97aff3ea81835835)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:59.168058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:59.174037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9243ff2-27b2-4843-b52d-b40b0382044f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:59.174037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9243ff2-27b2-4843-b52d-b40b0382044f service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:59.174037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9243ff2-27b2-4843-b52d-b40b0382044f service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:59.174037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9243ff2-27b2-4843-b52d-b40b0382044f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:59.179899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a9243ff2-27b2-4843-b52d-b40b0382044f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:59.286949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9243ff2-27b2-4843-b52d-b40b0382044f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:59.289404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9243ff2-27b2-4843-b52d-b40b0382044f service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:59.306809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 863/1714] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:59.310698 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2c596-0e95-4233-8b6d-0077d32e6cdc None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:59.310949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2c596-0e95-4233-8b6d-0077d32e6cdc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:59.311392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2c596-0e95-4233-8b6d-0077d32e6cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:59.352134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b9e2c596-0e95-4233-8b6d-0077d32e6cdc None None] MFA Rules not processed for user `9f03ab4adbba48eb9ac3d8540a7025b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:21:59.355464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b9e2c596-0e95-4233-8b6d-0077d32e6cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:59.413728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:59.436443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415a32d179144aeb97aff3ea81835835'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:59.448880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 864/1715] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:21:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:21:59.449880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2c596-0e95-4233-8b6d-0077d32e6cdc None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:59.451412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b9e2c596-0e95-4233-8b6d-0077d32e6cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:59.468825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415a32d179144aeb97aff3ea81835835'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:59.473738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2c596-0e95-4233-8b6d-0077d32e6cdc None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:59.487404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415a32d179144aeb97aff3ea81835835'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:59.488709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415a32d179144aeb97aff3ea81835835'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:59.489538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76441266-77ca-4bb7-b9fb-83435d087e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415a32d179144aeb97aff3ea81835835'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:59.492104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:59.492884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:59.493072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:59.493343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:59.498596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:59.570647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 852/1716] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:21:58 2022] DELETE /identity/v3/users/415a32d179144aeb97aff3ea81835835 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:21:59.570647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403c5d0-2c56-455a-96a9-119b8ccf487a None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:59.570647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403c5d0-2c56-455a-96a9-119b8ccf487a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:59.570942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403c5d0-2c56-455a-96a9-119b8ccf487a None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:59.580102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 853/1717] 10.222.0.30 () {54 vars in 961 bytes} [Sun Aug 7 12:21:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:21:59.581617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403c5d0-2c56-455a-96a9-119b8ccf487a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:59.592673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3403c5d0-2c56-455a-96a9-119b8ccf487a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:59.634239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:21:59.664229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:59.667931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:59.699867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 865/1718] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:21:59.702418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:21:59.706472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:21:59.718122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403c5d0-2c56-455a-96a9-119b8ccf487a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:59.725583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:59.726244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:59.726554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:59.726803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] PATH_INFO: `/v3/projects/6e1abf183aaf46f9beae2566b219e861` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:59.728939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1abf183aaf46f9beae2566b219e861)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:21:59.731038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:21:59.841189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23212-8a8c-4b3c-ba59-dbd15aa8fcac service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:21:59.897698 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1abf183aaf46f9beae2566b219e861', 'request_id': 'req-b3f595e8-a1d2-4d68-9654-5a7641366c6b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:21:59.991685 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35c460-ae25-4217-a230-817a146e8173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:21:59.993568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35c460-ae25-4217-a230-817a146e8173 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:21:59.993806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35c460-ae25-4217-a230-817a146e8173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:21:59.994083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35c460-ae25-4217-a230-817a146e8173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:21:59.999626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1abf183aaf46f9beae2566b219e861', 'request_id': 'req-b3f595e8-a1d2-4d68-9654-5a7641366c6b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.008604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1abf183aaf46f9beae2566b219e861', 'request_id': 'req-b3f595e8-a1d2-4d68-9654-5a7641366c6b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.014268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7e35c460-ae25-4217-a230-817a146e8173 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.022593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] The token cache is being invalidate because project 6e1abf183aaf46f9beae2566b219e861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:00.023690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.024816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b3f595e8-a1d2-4d68-9654-5a7641366c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.027077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 854/1719] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:21:59 2022] DELETE /identity/v3/projects/6e1abf183aaf46f9beae2566b219e861 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:00.028396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcbc0-dd2d-4b86-8448-c56526b687f9 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.028679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcbc0-dd2d-4b86-8448-c56526b687f9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.028951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcbc0-dd2d-4b86-8448-c56526b687f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.052145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e4ddcbc0-dd2d-4b86-8448-c56526b687f9 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:00.053570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e4ddcbc0-dd2d-4b86-8448-c56526b687f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.138464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 855/1720] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:00.138464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f16e6-cff6-4763-970e-4795d454058c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.139888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f16e6-cff6-4763-970e-4795d454058c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.140223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f16e6-cff6-4763-970e-4795d454058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.161467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35c460-ae25-4217-a230-817a146e8173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:00.163599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35c460-ae25-4217-a230-817a146e8173 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:00.185575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4e8f16e6-cff6-4763-970e-4795d454058c None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:00.186794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4e8f16e6-cff6-4763-970e-4795d454058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.187124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 866/1721] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:21:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:00.189139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20f13e-3299-4fd9-b04d-b78d6d65e9f4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.189429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20f13e-3299-4fd9-b04d-b78d6d65e9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.189701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20f13e-3299-4fd9-b04d-b78d6d65e9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.213464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4a20f13e-3299-4fd9-b04d-b78d6d65e9f4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:00.221363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4a20f13e-3299-4fd9-b04d-b78d6d65e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.234056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 867/1722] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:00.235085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20f13e-3299-4fd9-b04d-b78d6d65e9f4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:00.237484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4a20f13e-3299-4fd9-b04d-b78d6d65e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.338719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 856/1723] 10.222.0.30 () {58 vars in 1058 bytes} [Sun Aug 7 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:00.340824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9755bb8-e66c-4e75-801e-d7efd7fa478b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.341115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9755bb8-e66c-4e75-801e-d7efd7fa478b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.341388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9755bb8-e66c-4e75-801e-d7efd7fa478b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.370052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f9755bb8-e66c-4e75-801e-d7efd7fa478b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:00.371547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f9755bb8-e66c-4e75-801e-d7efd7fa478b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.375773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20f13e-3299-4fd9-b04d-b78d6d65e9f4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:00.376817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:00.381655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956d8b9-7f30-4f88-be6e-fe682a903353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:00.382683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d8b9-7f30-4f88-be6e-fe682a903353 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.383290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d8b9-7f30-4f88-be6e-fe682a903353 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.383395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 857/1724] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:00.384293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d8b9-7f30-4f88-be6e-fe682a903353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.384833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9755bb8-e66c-4e75-801e-d7efd7fa478b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:00.386465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f9755bb8-e66c-4e75-801e-d7efd7fa478b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.399448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956d8b9-7f30-4f88-be6e-fe682a903353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:00.401305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9755bb8-e66c-4e75-801e-d7efd7fa478b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:00.402390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956d8b9-7f30-4f88-be6e-fe682a903353 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:00.406807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:00.407600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.407903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.408149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] PATH_INFO: `/v3/projects/4c4f5478edba4618864db2664e93c0c6` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.411388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c4f5478edba4618864db2664e93c0c6)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:00.412739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:00.425415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 868/1725] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:00.437183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956d8b9-7f30-4f88-be6e-fe682a903353 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:00.455534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956d8b9-7f30-4f88-be6e-fe682a903353 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:00.465918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:00.466836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.467152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.467461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.469902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.484535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:00.487166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:00.547758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 869/1726] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:00.551836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:00.561760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c4f5478edba4618864db2664e93c0c6', 'request_id': 'req-45705b9f-5959-4921-9819-0c90ca222f27'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.575952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8a765-3bc9-41dd-bab2-e1de03eacfb3 service placement] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:00.701822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c4f5478edba4618864db2664e93c0c6', 'request_id': 'req-45705b9f-5959-4921-9819-0c90ca222f27'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.712548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c4f5478edba4618864db2664e93c0c6', 'request_id': 'req-45705b9f-5959-4921-9819-0c90ca222f27'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.724186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f4218-b618-451c-8468-50743a770d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:00.726023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4218-b618-451c-8468-50743a770d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.726287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4218-b618-451c-8468-50743a770d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.730517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4218-b618-451c-8468-50743a770d3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.733152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f4218-b618-451c-8468-50743a770d3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:00.733478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f4218-b618-451c-8468-50743a770d3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:00.742060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f4218-b618-451c-8468-50743a770d3e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:00.747640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] The token cache is being invalidate because project 4c4f5478edba4618864db2664e93c0c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:00.747952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.749201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:00.770074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 858/1727] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:00 2022] DELETE /identity/v3/projects/4c4f5478edba4618864db2664e93c0c6 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:00.770074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:00.773857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.785260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 870/1728] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:00.789188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5b0c4-83a6-4677-8df4-7b07b83bad92 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.789459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5b0c4-83a6-4677-8df4-7b07b83bad92 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.789737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5b0c4-83a6-4677-8df4-7b07b83bad92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.805995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45705b9f-5959-4921-9819-0c90ca222f27 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:00.820673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:00.821337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.829850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.829850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] PATH_INFO: `/v3/users/d2989133ec404c3ea79ce4efcaf5b9bd` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.833268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2989133ec404c3ea79ce4efcaf5b9bd)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:00.841327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:00.859575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-eac5b0c4-83a6-4677-8df4-7b07b83bad92 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:00.862351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eac5b0c4-83a6-4677-8df4-7b07b83bad92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.892534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 871/1729] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:00.897551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5b0c4-83a6-4677-8df4-7b07b83bad92 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:00.903936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eac5b0c4-83a6-4677-8df4-7b07b83bad92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:00.929398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5b0c4-83a6-4677-8df4-7b07b83bad92 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:00.936339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21737c29-de39-4f8e-81e6-87ffd0dfd68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:00.937148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21737c29-de39-4f8e-81e6-87ffd0dfd68a admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:00.937536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21737c29-de39-4f8e-81e6-87ffd0dfd68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:00.937917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21737c29-de39-4f8e-81e6-87ffd0dfd68a admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:00.938709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21737c29-de39-4f8e-81e6-87ffd0dfd68a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:00.940997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21737c29-de39-4f8e-81e6-87ffd0dfd68a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:01.104480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 872/1730] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:00 2022] POST /identity/v3/projects => generated 405 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:01.105825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930c67-df50-4349-8191-f3d1d390c5a6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:01.106070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930c67-df50-4349-8191-f3d1d390c5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:01.106347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930c67-df50-4349-8191-f3d1d390c5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:01.188196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2989133ec404c3ea79ce4efcaf5b9bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:01.225440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2989133ec404c3ea79ce4efcaf5b9bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:01.236594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a6930c67-df50-4349-8191-f3d1d390c5a6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:01.242846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6930c67-df50-4349-8191-f3d1d390c5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:01.254756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2989133ec404c3ea79ce4efcaf5b9bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:01.255817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2989133ec404c3ea79ce4efcaf5b9bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:01.257577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2989133ec404c3ea79ce4efcaf5b9bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:01.417897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 859/1731] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:00 2022] DELETE /identity/v3/users/d2989133ec404c3ea79ce4efcaf5b9bd => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:01.419061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:01.420652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:01.435684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 873/1732] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:01.438024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283910e-5929-403c-9de5-6755a6c944fe None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:01.438482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283910e-5929-403c-9de5-6755a6c944fe None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:01.439092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283910e-5929-403c-9de5-6755a6c944fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:01.464110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8283910e-5929-403c-9de5-6755a6c944fe None None] MFA Rules not processed for user `90dc7b751dd14c6f80205e709826423b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:01.465776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8283910e-5929-403c-9de5-6755a6c944fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:01.536493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:01.576891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626b4de-3d29-4317-b1fd-1e0cad87ee49 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:01.583166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:01.584375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:01.584706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:01.585048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:01.585657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:01.587766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:01.625599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 874/1733] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:01.640637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9a036-78a7-4062-ab9f-94a93695eb4f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:01.640932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9a036-78a7-4062-ab9f-94a93695eb4f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:01.641211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9a036-78a7-4062-ab9f-94a93695eb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:01.694606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-65f9a036-78a7-4062-ab9f-94a93695eb4f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:01.695875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-65f9a036-78a7-4062-ab9f-94a93695eb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:01.708272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 875/1734] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:01.709644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9a036-78a7-4062-ab9f-94a93695eb4f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:01.720449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9a036-78a7-4062-ab9f-94a93695eb4f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:01.729298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 860/1735] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:01 2022] POST /identity/v3/users => generated 303 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:01.738586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:01.745308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:01.745934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:01.746213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:01.746482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:01.746681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:01.749941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:01.749941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:01.753473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:01.778746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f2f5b-720b-4f99-a389-2011b933d70a admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:01.782812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 876/1736] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:01.783775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:01.785846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:01.796545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:01.801530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:01.802036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:01.816104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] PATH_INFO: `/v3/projects/33cefdfac4ed4cc4b241301596ce4cd8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:01.820906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cefdfac4ed4cc4b241301596ce4cd8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:01.829083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:01.917782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb8cf0-0d1f-486d-a655-cc7c9a53af76 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:01.932255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6e187-5318-466e-8bcb-a8ce9d88625d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:01.933873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6e187-5318-466e-8bcb-a8ce9d88625d service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:01.934175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6e187-5318-466e-8bcb-a8ce9d88625d service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:01.934379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6e187-5318-466e-8bcb-a8ce9d88625d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:01.961003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6e187-5318-466e-8bcb-a8ce9d88625d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:01.962461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6e187-5318-466e-8bcb-a8ce9d88625d service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:01.995051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 877/1737] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:02.003384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6e187-5318-466e-8bcb-a8ce9d88625d service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.030606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cefdfac4ed4cc4b241301596ce4cd8', 'request_id': 'req-b96234ef-13c3-4df1-81e5-80f9609bc0ea'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:02.042935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6e187-5318-466e-8bcb-a8ce9d88625d service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:02.125652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cefdfac4ed4cc4b241301596ce4cd8', 'request_id': 'req-b96234ef-13c3-4df1-81e5-80f9609bc0ea'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:02.139372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cefdfac4ed4cc4b241301596ce4cd8', 'request_id': 'req-b96234ef-13c3-4df1-81e5-80f9609bc0ea'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:02.170375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] The token cache is being invalidate because project 33cefdfac4ed4cc4b241301596ce4cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:02.170845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:02.179424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:02.183493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 861/1738] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:01 2022] DELETE /identity/v3/projects/33cefdfac4ed4cc4b241301596ce4cd8 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:02.188247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.190481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:02.233582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654218-ed6c-45d7-9eed-a2ae2327e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:02.234774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654218-ed6c-45d7-9eed-a2ae2327e882 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.234774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654218-ed6c-45d7-9eed-a2ae2327e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.234981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654218-ed6c-45d7-9eed-a2ae2327e882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.236875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654218-ed6c-45d7-9eed-a2ae2327e882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:02.248243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654218-ed6c-45d7-9eed-a2ae2327e882 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:02.305334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:02.309435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96234ef-13c3-4df1-81e5-80f9609bc0ea admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:02.314868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 878/1739] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:02.319994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36476-bf0f-4836-897d-9d8cf0008c06 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.320304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36476-bf0f-4836-897d-9d8cf0008c06 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.321694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36476-bf0f-4836-897d-9d8cf0008c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.321877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f00e83-837a-4dd7-8229-87838b0056de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:02.322749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f00e83-837a-4dd7-8229-87838b0056de admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.323227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f00e83-837a-4dd7-8229-87838b0056de admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.323791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f00e83-837a-4dd7-8229-87838b0056de admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.324623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f00e83-837a-4dd7-8229-87838b0056de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:02.326805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f00e83-837a-4dd7-8229-87838b0056de admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:02.374955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-afc36476-bf0f-4836-897d-9d8cf0008c06 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:02.377114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-afc36476-bf0f-4836-897d-9d8cf0008c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:02.418389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 879/1740] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:02.420996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc36476-bf0f-4836-897d-9d8cf0008c06 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.421856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-afc36476-bf0f-4836-897d-9d8cf0008c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:02.436771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc36476-bf0f-4836-897d-9d8cf0008c06 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:02.442682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43705a75-c88b-4421-9ffa-9a72feb137d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:02.444233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43705a75-c88b-4421-9ffa-9a72feb137d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.444718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43705a75-c88b-4421-9ffa-9a72feb137d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.456865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43705a75-c88b-4421-9ffa-9a72feb137d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.456865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43705a75-c88b-4421-9ffa-9a72feb137d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:02.456865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43705a75-c88b-4421-9ffa-9a72feb137d3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:02.457147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 862/1741] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:02 2022] POST /identity/v3/projects => generated 379 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:02.457147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623394a-87a3-43de-84ba-aa7c33869b88 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.457147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623394a-87a3-43de-84ba-aa7c33869b88 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.457147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623394a-87a3-43de-84ba-aa7c33869b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.468745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 880/1742] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:02.470264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43705a75-c88b-4421-9ffa-9a72feb137d3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.485259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43705a75-c88b-4421-9ffa-9a72feb137d3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:02.516677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a623394a-87a3-43de-84ba-aa7c33869b88 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:02.521636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a623394a-87a3-43de-84ba-aa7c33869b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:02.561640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2007c-b30b-4d14-b622-2fbc904b9fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:02.562273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2007c-b30b-4d14-b622-2fbc904b9fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.562558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2007c-b30b-4d14-b622-2fbc904b9fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.562835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2007c-b30b-4d14-b622-2fbc904b9fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.563509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2007c-b30b-4d14-b622-2fbc904b9fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:02.565724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2007c-b30b-4d14-b622-2fbc904b9fd4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:02.654765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 863/1743] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:02.667448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 881/1744] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:02 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:02.671498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623394a-87a3-43de-84ba-aa7c33869b88 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.673542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ba49-e2bd-421b-955d-24cfda795d25 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.673843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ba49-e2bd-421b-955d-24cfda795d25 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.674128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ba49-e2bd-421b-955d-24cfda795d25 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.675989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 882/1745] 10.222.0.30 () {54 vars in 912 bytes} [Sun Aug 7 12:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:22:02.676775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773ba49-e2bd-421b-955d-24cfda795d25 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.692420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773ba49-e2bd-421b-955d-24cfda795d25 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:02.699366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:02.699988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.700274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.700552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.701217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:02.701525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:02.703951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:02.704886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623394a-87a3-43de-84ba-aa7c33869b88 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:02.718868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f184f-cc10-4100-ad63-7d739f5e7918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:02.719744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f184f-cc10-4100-ad63-7d739f5e7918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.720161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f184f-cc10-4100-ad63-7d739f5e7918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.720519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f184f-cc10-4100-ad63-7d739f5e7918 admin admin] PATH_INFO: `/v3/projects/dd487a44183d4e67a4e7d3694f7a5dad/users/a858cdde46ec4bf8a0ef98f72f363be3/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.730029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 883/1746] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:02.731047 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.743066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec779595-3ba7-4330-9eb6-9bcf93ee993d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:02.750718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b49d08-fded-45bf-a57c-9ca3d2ddb2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:02.751382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b49d08-fded-45bf-a57c-9ca3d2ddb2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.751667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b49d08-fded-45bf-a57c-9ca3d2ddb2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.752107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b49d08-fded-45bf-a57c-9ca3d2ddb2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.752906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b49d08-fded-45bf-a57c-9ca3d2ddb2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:02.754952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b49d08-fded-45bf-a57c-9ca3d2ddb2f1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:02.755913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f184f-cc10-4100-ad63-7d739f5e7918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd487a44183d4e67a4e7d3694f7a5dad, user_id=a858cdde46ec4bf8a0ef98f72f363be3, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:02.762716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f184f-cc10-4100-ad63-7d739f5e7918 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:02.770450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 884/1747] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:02.772555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae422eb0-7b65-425e-bfdf-3b133f8b8762 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:02.772845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae422eb0-7b65-425e-bfdf-3b133f8b8762 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:02.773312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae422eb0-7b65-425e-bfdf-3b133f8b8762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:02.797892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ae422eb0-7b65-425e-bfdf-3b133f8b8762 None None] MFA Rules not processed for user `f7aa31ea778140189b041939e3f01aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:02.799120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ae422eb0-7b65-425e-bfdf-3b133f8b8762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:02.836330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 864/1748] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:02 2022] PUT /identity/v3/projects/dd487a44183d4e67a4e7d3694f7a5dad/users/a858cdde46ec4bf8a0ef98f72f363be3/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:02.837973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f184f-cc10-4100-ad63-7d739f5e7918 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.885471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 885/1749] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:02.886120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae422eb0-7b65-425e-bfdf-3b133f8b8762 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:02.904304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae422eb0-7b65-425e-bfdf-3b133f8b8762 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:03.017911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a81891-5656-4689-acfb-b94f434ab9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:03.018547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a81891-5656-4689-acfb-b94f434ab9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:03.018832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a81891-5656-4689-acfb-b94f434ab9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:03.019150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a81891-5656-4689-acfb-b94f434ab9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:03.019815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a81891-5656-4689-acfb-b94f434ab9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:03.022081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a81891-5656-4689-acfb-b94f434ab9ca admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:03.031378 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f184f-cc10-4100-ad63-7d739f5e7918 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:03.040981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6e842-2db1-470a-bea3-5421fb6d0b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:03.041852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6e842-2db1-470a-bea3-5421fb6d0b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:03.042337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6e842-2db1-470a-bea3-5421fb6d0b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:03.043246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 886/1750] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:03.044872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93e38a-96e6-4d3b-9b5b-a6a3ebd6f788 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:03.045166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93e38a-96e6-4d3b-9b5b-a6a3ebd6f788 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:03.045447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93e38a-96e6-4d3b-9b5b-a6a3ebd6f788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:03.046150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6e842-2db1-470a-bea3-5421fb6d0b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:03.047578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6e842-2db1-470a-bea3-5421fb6d0b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:03.066439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6e842-2db1-470a-bea3-5421fb6d0b2e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:03.082607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-da93e38a-96e6-4d3b-9b5b-a6a3ebd6f788 None None] MFA Rules not processed for user `a858cdde46ec4bf8a0ef98f72f363be3`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:03.086026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-da93e38a-96e6-4d3b-9b5b-a6a3ebd6f788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:03.195879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 887/1751] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:03.196948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93e38a-96e6-4d3b-9b5b-a6a3ebd6f788 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:03.198431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-da93e38a-96e6-4d3b-9b5b-a6a3ebd6f788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:03.217015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 865/1752] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:02 2022] POST /identity/v3/projects => generated 373 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:03.218004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6e842-2db1-470a-bea3-5421fb6d0b2e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:03.229648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93e38a-96e6-4d3b-9b5b-a6a3ebd6f788 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:03.381166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:03.386759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:03.387294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:03.387643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] PATH_INFO: `/v3/projects/40bb9b119a7744f9a96ea8453e0850ac` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:03.389070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40bb9b119a7744f9a96ea8453e0850ac)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:03.400073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:03.432540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:03.460575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6e842-2db1-470a-bea3-5421fb6d0b2e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:03.474071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:03.477207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:03.477665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:03.478083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] PATH_INFO: `/v3/projects/7dd9359b8d724ec9ac0fdeeb305ee352/users/15d895ea7f484146acf9e97c749c93af/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:03.504116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd9359b8d724ec9ac0fdeeb305ee352, user_id=15d895ea7f484146acf9e97c749c93af, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:03.512850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:03.648977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 866/1753] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:03 2022] PUT /identity/v3/projects/7dd9359b8d724ec9ac0fdeeb305ee352/users/15d895ea7f484146acf9e97c749c93af/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:03.651421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:03.656427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:03.702099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40bb9b119a7744f9a96ea8453e0850ac', 'request_id': 'req-2aae28ea-d871-4af6-b475-166bc64ac0b9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:03.806288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74b97d-8807-437a-98b7-861c965bf151 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:03.822730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40bb9b119a7744f9a96ea8453e0850ac', 'request_id': 'req-2aae28ea-d871-4af6-b475-166bc64ac0b9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:03.837325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:03.837661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40bb9b119a7744f9a96ea8453e0850ac', 'request_id': 'req-2aae28ea-d871-4af6-b475-166bc64ac0b9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:03.868798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] The token cache is being invalidate because project 40bb9b119a7744f9a96ea8453e0850ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:03.869135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:03.874367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:03.879416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 888/1754] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:03 2022] DELETE /identity/v3/projects/40bb9b119a7744f9a96ea8453e0850ac => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:03.879501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:03.882586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:03.910290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:03.910913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:03.911230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:03.911504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:03.916581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:03.936881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:03.938403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:03.955020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 867/1755] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:03.955020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:03.955020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:04.081720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae28ea-d871-4af6-b475-166bc64ac0b9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:04.087453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622604a-0da4-4ecd-8f3b-81fecfdc6544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:04.088320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622604a-0da4-4ecd-8f3b-81fecfdc6544 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:04.088605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622604a-0da4-4ecd-8f3b-81fecfdc6544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:04.088878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622604a-0da4-4ecd-8f3b-81fecfdc6544 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:04.091445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622604a-0da4-4ecd-8f3b-81fecfdc6544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:04.095479 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622604a-0da4-4ecd-8f3b-81fecfdc6544 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:04.134946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641efc09-a29b-41db-a504-727ed67a4861 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:04.167212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:04.170742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:04.170742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:04.170742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:04.170742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:04.178216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:04.236234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 889/1756] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:03 2022] POST /identity/v3/projects => generated 409 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:04.236234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04797-d697-47bb-b589-55125f82c0f4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:04.236234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04797-d697-47bb-b589-55125f82c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:04.236234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04797-d697-47bb-b589-55125f82c0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:04.269202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-efa04797-d697-47bb-b589-55125f82c0f4 None None] MFA Rules not processed for user `15d895ea7f484146acf9e97c749c93af`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:04.272151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-efa04797-d697-47bb-b589-55125f82c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:04.298441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 868/1757] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:03 2022] POST /identity/v3/users => generated 287 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:04.301027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:04.304874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:04.327073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e74e81-273a-4d54-9402-89662504d2cc admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:04.445905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 890/1758] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:04.447405 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa04797-d697-47bb-b589-55125f82c0f4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:04.513840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:04.516134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:04.517706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:04.518765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] PATH_INFO: `/v3/users/99470ebdf01a444691a1b0422f00c644` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:04.523513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99470ebdf01a444691a1b0422f00c644)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:04.527302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:04.590093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa04797-d697-47bb-b589-55125f82c0f4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:04.594778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2e3be-f2bf-4cac-b862-66a2bd75ccb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:04.595593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2e3be-f2bf-4cac-b862-66a2bd75ccb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:04.595880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2e3be-f2bf-4cac-b862-66a2bd75ccb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:04.596160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2e3be-f2bf-4cac-b862-66a2bd75ccb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:04.596804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2e3be-f2bf-4cac-b862-66a2bd75ccb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:04.598796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2e3be-f2bf-4cac-b862-66a2bd75ccb8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:04.708901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 891/1759] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:04 2022] POST /identity/v3/users => generated 311 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:04.710186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2e3be-f2bf-4cac-b862-66a2bd75ccb8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:04.733070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2e3be-f2bf-4cac-b862-66a2bd75ccb8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:04.753378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:04.754528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:04.755120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:04.756554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:04.757281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99470ebdf01a444691a1b0422f00c644'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:04.759553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:04.762106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:04.784911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 892/1760] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:04.786167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:04.788029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99470ebdf01a444691a1b0422f00c644'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:04.793008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:04.802679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99470ebdf01a444691a1b0422f00c644'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:04.806474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99470ebdf01a444691a1b0422f00c644'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:04.807276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99470ebdf01a444691a1b0422f00c644'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:04.816424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b722-70a9-4bb9-8514-82936495e264 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:04.833539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:04.835309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:04.835607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:04.835891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:04.836549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:04.838498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:04.916555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 869/1761] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:04 2022] DELETE /identity/v3/users/99470ebdf01a444691a1b0422f00c644 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:04.917621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:04.932773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:04.949885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 893/1762] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:04 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:04.963340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:04.965887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:05.096525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ced5-bc98-45e9-9c15-3aacfb0f7e8d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:05.125660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:05.126420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.126767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.127240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.128095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:05.129828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:05.143371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44782b-8e78-48b9-8cf5-c5f0eaf8c360 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:05.159218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:05.160202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.160628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.161075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.163198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:05.165560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:05.186592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 870/1763] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:05.188339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:05.197400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:05.199896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 894/1764] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:05.210629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:05.215997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:05.216748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe9517-533c-4a23-8602-ada8e0dac450 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:05.246306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82875b6-2842-4c16-9803-ccca79a97aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:05.247900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82875b6-2842-4c16-9803-ccca79a97aed admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.249758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82875b6-2842-4c16-9803-ccca79a97aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.251447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82875b6-2842-4c16-9803-ccca79a97aed admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.253572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82875b6-2842-4c16-9803-ccca79a97aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:05.255626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82875b6-2842-4c16-9803-ccca79a97aed admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:05.262445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c863-664d-4dc4-a491-9061096371b4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:05.280056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:05.283949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:05.285055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.285170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.285515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] PATH_INFO: `/v3/users/e2bfd7b11a1349da9edb69744bcb4279` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.287587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bfd7b11a1349da9edb69744bcb4279)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:05.289696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:05.347761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 871/1765] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:05 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:05.347761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82875b6-2842-4c16-9803-ccca79a97aed admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:05.361601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82875b6-2842-4c16-9803-ccca79a97aed admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:05.367533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3557219-0a1b-451a-b40a-a762718a59c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:05.368209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3557219-0a1b-451a-b40a-a762718a59c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.368689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3557219-0a1b-451a-b40a-a762718a59c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.369146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3557219-0a1b-451a-b40a-a762718a59c4 admin admin] PATH_INFO: `/v3/projects/036a12fe0a6540fbaa3067164720e8a9/users/5258fe7f2ac34691859ee5da42fbf7c8/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.391108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3557219-0a1b-451a-b40a-a762718a59c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036a12fe0a6540fbaa3067164720e8a9, user_id=5258fe7f2ac34691859ee5da42fbf7c8, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:05.394182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3557219-0a1b-451a-b40a-a762718a59c4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:05.405421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:05.536294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bfd7b11a1349da9edb69744bcb4279'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:05.538693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 872/1766] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:05 2022] PUT /identity/v3/projects/036a12fe0a6540fbaa3067164720e8a9/users/5258fe7f2ac34691859ee5da42fbf7c8/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:05.539822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3557219-0a1b-451a-b40a-a762718a59c4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:05.567649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bfd7b11a1349da9edb69744bcb4279'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:05.580620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bfd7b11a1349da9edb69744bcb4279'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:05.581373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bfd7b11a1349da9edb69744bcb4279'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:05.582178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bfd7b11a1349da9edb69744bcb4279'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:05.620015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3557219-0a1b-451a-b40a-a762718a59c4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:05.625103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a52f82-dc5e-47fd-8638-40344546b9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:05.625841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52f82-dc5e-47fd-8638-40344546b9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.626118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52f82-dc5e-47fd-8638-40344546b9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.626395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52f82-dc5e-47fd-8638-40344546b9b2 admin admin] PATH_INFO: `/v3/projects/df2d537ee6044619b90d744ac258b4b1/users/bc6a6aec1a554b81927d28bf86df964e/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.656193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a52f82-dc5e-47fd-8638-40344546b9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2d537ee6044619b90d744ac258b4b1, user_id=bc6a6aec1a554b81927d28bf86df964e, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:05.660065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a52f82-dc5e-47fd-8638-40344546b9b2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:05.661494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 895/1767] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:05 2022] DELETE /identity/v3/users/e2bfd7b11a1349da9edb69744bcb4279 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:05.664263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:05.668693 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:05.716007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:05.747311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 873/1768] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:05 2022] PUT /identity/v3/projects/df2d537ee6044619b90d744ac258b4b1/users/bc6a6aec1a554b81927d28bf86df964e/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:05.749075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d830-ba26-429d-98b8-5a98314aa922 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.749075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d830-ba26-429d-98b8-5a98314aa922 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.749285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d830-ba26-429d-98b8-5a98314aa922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.766563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3301ae-a69e-4ebf-b50b-e7d715a553b1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:05.792803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0e38d830-ba26-429d-98b8-5a98314aa922 None None] MFA Rules not processed for user `c8fb20cfc3b6479198059b4383f47527`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:05.796430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0e38d830-ba26-429d-98b8-5a98314aa922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:05.856561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:05.857303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.858551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.858551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.858551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:05.860335 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:05.870502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 874/1769] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:05.871677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc3623-f133-420a-a8f3-600064d235d9 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.872039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc3623-f133-420a-a8f3-600064d235d9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.872426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc3623-f133-420a-a8f3-600064d235d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.874180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 896/1770] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:05.876260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:05.892751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:05.942642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-65dc3623-f133-420a-a8f3-600064d235d9 None None] MFA Rules not processed for user `5258fe7f2ac34691859ee5da42fbf7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:05.945080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-65dc3623-f133-420a-a8f3-600064d235d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:05.949966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fef3c-a60c-4356-8437-46aa4b93d5d5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:05.965702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:05.966362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:05.968084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:05.968478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] PATH_INFO: `/v3/projects/d1da10c585b5418382f58dab93b46365` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:05.971517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1da10c585b5418382f58dab93b46365)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:05.976450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:06.108704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:06.128451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 875/1771] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:06.130370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9cea6-33d3-4d62-bc0f-81b47534a954 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.130945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9cea6-33d3-4d62-bc0f-81b47534a954 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.131390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9cea6-33d3-4d62-bc0f-81b47534a954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.144615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1da10c585b5418382f58dab93b46365', 'request_id': 'req-eb88c758-b590-4b0e-ac8c-1a518f9711b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:06.192189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-b5b9cea6-33d3-4d62-bc0f-81b47534a954 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:06.195210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b5b9cea6-33d3-4d62-bc0f-81b47534a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.239136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1da10c585b5418382f58dab93b46365', 'request_id': 'req-eb88c758-b590-4b0e-ac8c-1a518f9711b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:06.247297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1da10c585b5418382f58dab93b46365', 'request_id': 'req-eb88c758-b590-4b0e-ac8c-1a518f9711b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:06.265256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] The token cache is being invalidate because project d1da10c585b5418382f58dab93b46365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:06.272643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 876/1772] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:06.273563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9cea6-33d3-4d62-bc0f-81b47534a954 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:06.274968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b5b9cea6-33d3-4d62-bc0f-81b47534a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.277719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:06.282467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:06.284923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 897/1773] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:05 2022] DELETE /identity/v3/projects/d1da10c585b5418382f58dab93b46365 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:06.285665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:06.287610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.296297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9cea6-33d3-4d62-bc0f-81b47534a954 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:06.302142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88c758-b590-4b0e-ac8c-1a518f9711b6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:06.304440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d64128-8b24-445e-9067-46766d1e72fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:06.305206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d64128-8b24-445e-9067-46766d1e72fe admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.305501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d64128-8b24-445e-9067-46766d1e72fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.305767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d64128-8b24-445e-9067-46766d1e72fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.306409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d64128-8b24-445e-9067-46766d1e72fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:06.308320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d64128-8b24-445e-9067-46766d1e72fe admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:06.309490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:06.310171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.310511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.311850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.311850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:06.313709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:06.325817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 877/1774] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:06.327293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab9084-6758-4430-9faf-13076bf771de None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.327581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab9084-6758-4430-9faf-13076bf771de None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.327859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab9084-6758-4430-9faf-13076bf771de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.343460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 898/1775] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:06.345489 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:06.350564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.368851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d9ab9084-6758-4430-9faf-13076bf771de None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:06.372306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d9ab9084-6758-4430-9faf-13076bf771de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.392255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 878/1776] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:06.393269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab9084-6758-4430-9faf-13076bf771de None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:06.394859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d9ab9084-6758-4430-9faf-13076bf771de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.415866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab9084-6758-4430-9faf-13076bf771de None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:06.425313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165b631-658f-4461-8626-0710272f3179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:06.425991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b631-658f-4461-8626-0710272f3179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.426271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b631-658f-4461-8626-0710272f3179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.426545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b631-658f-4461-8626-0710272f3179 admin admin] PATH_INFO: `/v3/projects/df2d537ee6044619b90d744ac258b4b1/users/bc6a6aec1a554b81927d28bf86df964e/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.429273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165b631-658f-4461-8626-0710272f3179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2d537ee6044619b90d744ac258b4b1, user_id=bc6a6aec1a554b81927d28bf86df964e, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:06.431706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165b631-658f-4461-8626-0710272f3179 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:06.447232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 879/1777] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:06 2022] PUT /identity/v3/projects/df2d537ee6044619b90d744ac258b4b1/users/bc6a6aec1a554b81927d28bf86df964e/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:06.447232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165b631-658f-4461-8626-0710272f3179 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:06.485127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba4a19-27d1-40cb-815a-5eb4254844ef admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:06.591171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:06.591796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165b631-658f-4461-8626-0710272f3179 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:06.592054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.592205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.592484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.594370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.596395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d2463-0686-4294-93e4-0ac3d71ea18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:06.597722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d2463-0686-4294-93e4-0ac3d71ea18b admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.597722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d2463-0686-4294-93e4-0ac3d71ea18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.597722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d2463-0686-4294-93e4-0ac3d71ea18b admin admin] PATH_INFO: `/v3/projects/683e0273c73e4a758cda5762fb80682d/users/4671ed776399454f811e08832fc712f3/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.615351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d2463-0686-4294-93e4-0ac3d71ea18b admin admin] RBAC: Authorizing `identity:create_grant(project_id=683e0273c73e4a758cda5762fb80682d, user_id=4671ed776399454f811e08832fc712f3, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:06.619214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d2463-0686-4294-93e4-0ac3d71ea18b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:06.689547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:06.691973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:06.702459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 899/1778] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:06.703216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:06.704553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.730717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 880/1779] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:06 2022] PUT /identity/v3/projects/683e0273c73e4a758cda5762fb80682d/users/4671ed776399454f811e08832fc712f3/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:06.730717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd29289-bdb3-4b07-a11a-d45d1ee8ef6e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.730717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd29289-bdb3-4b07-a11a-d45d1ee8ef6e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.730717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd29289-bdb3-4b07-a11a-d45d1ee8ef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.774005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-dcd29289-bdb3-4b07-a11a-d45d1ee8ef6e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:06.775435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-dcd29289-bdb3-4b07-a11a-d45d1ee8ef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.782631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:06.783896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dffb7-7c77-456c-8604-c1dc77291b51 service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:06.848571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa590365-7559-4504-a4d6-d0b7ede26ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:06.849675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa590365-7559-4504-a4d6-d0b7ede26ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.850149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa590365-7559-4504-a4d6-d0b7ede26ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.850772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa590365-7559-4504-a4d6-d0b7ede26ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.853662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fa590365-7559-4504-a4d6-d0b7ede26ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.858855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 881/1780] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:06.860310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7fc62-7589-432d-8782-fa4f8252deb0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.868466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7fc62-7589-432d-8782-fa4f8252deb0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.868723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7fc62-7589-432d-8782-fa4f8252deb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.871040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa590365-7559-4504-a4d6-d0b7ede26ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:06.872895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa590365-7559-4504-a4d6-d0b7ede26ed4 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:06.899845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2fd7fc62-7589-432d-8782-fa4f8252deb0 None None] MFA Rules not processed for user `bc6a6aec1a554b81927d28bf86df964e`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:06.901075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2fd7fc62-7589-432d-8782-fa4f8252deb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:06.904579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 900/1781] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:06.906227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19119aac-d770-4a42-843b-6ef94cac5428 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:06.906834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19119aac-d770-4a42-843b-6ef94cac5428 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:06.907509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19119aac-d770-4a42-843b-6ef94cac5428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:06.960646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:06.991262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-19119aac-d770-4a42-843b-6ef94cac5428 None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:06.995616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-19119aac-d770-4a42-843b-6ef94cac5428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:07.042977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 882/1782] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:07.042977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492309d8-4480-40a3-95c2-6ab3d1b99fcb None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:07.047741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492309d8-4480-40a3-95c2-6ab3d1b99fcb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:07.048056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492309d8-4480-40a3-95c2-6ab3d1b99fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:07.101550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-492309d8-4480-40a3-95c2-6ab3d1b99fcb None None] MFA Rules not processed for user `4671ed776399454f811e08832fc712f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:07.102819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-492309d8-4480-40a3-95c2-6ab3d1b99fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:07.112059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 901/1783] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:07.113136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19119aac-d770-4a42-843b-6ef94cac5428 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:07.137000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19119aac-d770-4a42-843b-6ef94cac5428 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:07.147697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:07.148326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:07.148662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:07.148947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:07.155477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:07.155856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:07.157611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:07.182384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 902/1784] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:07.183492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:07.185560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:07.258092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246a37b-e896-45f9-82f1-cd36b7c84cf1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:07.259895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a6d09-b1a7-47cf-9486-58b8dc7da239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:07.260520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6d09-b1a7-47cf-9486-58b8dc7da239 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:07.260802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6d09-b1a7-47cf-9486-58b8dc7da239 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:07.261076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6d09-b1a7-47cf-9486-58b8dc7da239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:07.281113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 883/1785] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:07.282344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492309d8-4480-40a3-95c2-6ab3d1b99fcb None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:07.282473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a6d09-b1a7-47cf-9486-58b8dc7da239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:07.284028 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a6d09-b1a7-47cf-9486-58b8dc7da239 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:07.299235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492309d8-4480-40a3-95c2-6ab3d1b99fcb None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:07.307681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:07.308648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:07.309179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:07.310484 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:07.311409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:07.313945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:07.333880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 903/1786] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:07 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:07.345151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a6d09-b1a7-47cf-9486-58b8dc7da239 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:07.378178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a6d09-b1a7-47cf-9486-58b8dc7da239 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:07.393351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0bb7-bff1-40d9-8753-ed1bb2c7457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:07.394542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0bb7-bff1-40d9-8753-ed1bb2c7457e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:07.395851 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0bb7-bff1-40d9-8753-ed1bb2c7457e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:07.400605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0bb7-bff1-40d9-8753-ed1bb2c7457e admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:07.400605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d0bb7-bff1-40d9-8753-ed1bb2c7457e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:07.413811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d0bb7-bff1-40d9-8753-ed1bb2c7457e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:07.449774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 884/1787] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:07 2022] POST /identity/v3/projects => generated 357 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:07.451414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:07.457637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:07.515718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 904/1788] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:07 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:07.516847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0bb7-bff1-40d9-8753-ed1bb2c7457e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:07.562510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:07.632783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0bb7-bff1-40d9-8753-ed1bb2c7457e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:07.660329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8e99d-ee42-4030-8f4a-01806392fd83 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:07.672355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ec238-bfc0-41a0-9faf-b641870b9417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:07.674603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ec238-bfc0-41a0-9faf-b641870b9417 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:07.676129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ec238-bfc0-41a0-9faf-b641870b9417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:07.676355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:07.676627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ec238-bfc0-41a0-9faf-b641870b9417 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:07.677561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ec238-bfc0-41a0-9faf-b641870b9417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:07.678009 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:07.678717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:07.679393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] PATH_INFO: `/v3/users/5258fe7f2ac34691859ee5da42fbf7c8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:07.681700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5258fe7f2ac34691859ee5da42fbf7c8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:07.684074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:07.688221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ec238-bfc0-41a0-9faf-b641870b9417 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:07.726865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:07.808279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 905/1789] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:07 2022] POST /identity/v3/users => generated 279 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:07.810719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ec238-bfc0-41a0-9faf-b641870b9417 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:07.828587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ec238-bfc0-41a0-9faf-b641870b9417 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:07.835240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:07.835449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:07.835993 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:07.836379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:07.837478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:07.840155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:07.854243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:07.979873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 906/1790] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:07 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:07.979873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:07.997138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.001463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5258fe7f2ac34691859ee5da42fbf7c8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:08.044871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5258fe7f2ac34691859ee5da42fbf7c8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:08.054392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5258fe7f2ac34691859ee5da42fbf7c8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:08.055312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5258fe7f2ac34691859ee5da42fbf7c8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:08.056024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5ea7d17c-78cd-4fc3-9b30-01ee823ca7c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5258fe7f2ac34691859ee5da42fbf7c8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:08.136380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 885/1791] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:07 2022] DELETE /identity/v3/users/5258fe7f2ac34691859ee5da42fbf7c8 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:08.137931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f4462-2aa9-4686-85d8-0da4ed36c115 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.139396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f4462-2aa9-4686-85d8-0da4ed36c115 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.139895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f4462-2aa9-4686-85d8-0da4ed36c115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.188480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-514f4462-2aa9-4686-85d8-0da4ed36c115 None None] MFA Rules not processed for user `f7aa31ea778140189b041939e3f01aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:08.191856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-514f4462-2aa9-4686-85d8-0da4ed36c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.203813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8285a-a8be-4ed0-9141-54732bf2b38e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.236440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:08.283258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 886/1792] 10.222.0.18 () {58 vars in 1015 bytes} [Sun Aug 7 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:08.285962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f4462-2aa9-4686-85d8-0da4ed36c115 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:08.289257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-514f4462-2aa9-4686-85d8-0da4ed36c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.365694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.367233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.367776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.368093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.370389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.385462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f4462-2aa9-4686-85d8-0da4ed36c115 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.390642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.391351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.391731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.392204 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.392874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:08.394954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:08.410259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 887/1793] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:08.411483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:08.414318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.454954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:08.456941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:08.481925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 907/1794] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:22:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:22:08.483307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:08.486905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.506643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31c0a2-b89c-4c98-8684-c708d843f869 service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.511894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db1a3b-da43-4046-a386-a96e2d2ddd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.512571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1a3b-da43-4046-a386-a96e2d2ddd83 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.513387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1a3b-da43-4046-a386-a96e2d2ddd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.513801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1a3b-da43-4046-a386-a96e2d2ddd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.514593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1a3b-da43-4046-a386-a96e2d2ddd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:08.517692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1a3b-da43-4046-a386-a96e2d2ddd83 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:08.540538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 908/1795] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:08.543306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96a37d-1be5-4c3a-a926-78ea314d4460 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.543600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96a37d-1be5-4c3a-a926-78ea314d4460 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.543883 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96a37d-1be5-4c3a-a926-78ea314d4460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.560008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601f490-a11f-4253-af71-551bbe102748 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.573271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.584804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:08.585220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.587724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.588050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.596002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.606585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8e96a37d-1be5-4c3a-a926-78ea314d4460 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:08.607739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8e96a37d-1be5-4c3a-a926-78ea314d4460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.629169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 909/1796] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:08.630325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96a37d-1be5-4c3a-a926-78ea314d4460 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:08.639716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8e96a37d-1be5-4c3a-a926-78ea314d4460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.669605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96a37d-1be5-4c3a-a926-78ea314d4460 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.677515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559410c-f968-4e8d-98fc-bc7c247b39e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.678911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559410c-f968-4e8d-98fc-bc7c247b39e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.681042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559410c-f968-4e8d-98fc-bc7c247b39e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.681438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559410c-f968-4e8d-98fc-bc7c247b39e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.694783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559410c-f968-4e8d-98fc-bc7c247b39e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:08.700045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559410c-f968-4e8d-98fc-bc7c247b39e9 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:08.714320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 910/1797] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:08.715429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559410c-f968-4e8d-98fc-bc7c247b39e9 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:08.744311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559410c-f968-4e8d-98fc-bc7c247b39e9 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.770318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.771435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.772255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.772845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.773722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:08.777307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:08.778499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:08.779979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:08.782044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:08.793309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 888/1798] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:08 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:08.803334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 911/1799] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:08.811100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:08.818671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.838329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:08.841615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:08.848658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8bbd9-4f58-49ab-9f4c-02cf7364dd5c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.860787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd60b5-955b-445e-a45c-5b87211f1ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.865531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd60b5-955b-445e-a45c-5b87211f1ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.865531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd60b5-955b-445e-a45c-5b87211f1ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.865531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd60b5-955b-445e-a45c-5b87211f1ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.877878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd60b5-955b-445e-a45c-5b87211f1ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:08.879696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd60b5-955b-445e-a45c-5b87211f1ee2 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:08.881115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f1f6-2c83-4ce9-90ac-4b722d6f2e1c service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.894066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.895121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.900390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.900874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] PATH_INFO: `/v3/projects/b004823ea9d24e7ab4f5f94860fdd04a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:08.902553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b004823ea9d24e7ab4f5f94860fdd04a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:08.904852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:08.908567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 912/1800] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:08.909557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd60b5-955b-445e-a45c-5b87211f1ee2 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:08.968526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd60b5-955b-445e-a45c-5b87211f1ee2 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:08.987352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da85165-9492-48be-8312-0d268bc8d502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:08.988201 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da85165-9492-48be-8312-0d268bc8d502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:08.988487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da85165-9492-48be-8312-0d268bc8d502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:08.988805 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da85165-9492-48be-8312-0d268bc8d502 admin admin] PATH_INFO: `/v3/projects/50a71b32fd0d495ba9e245428f351186/users/ed37a87cea5e4714883d8b5515a4523c/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:09.019221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da85165-9492-48be-8312-0d268bc8d502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a71b32fd0d495ba9e245428f351186, user_id=ed37a87cea5e4714883d8b5515a4523c, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:09.022244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da85165-9492-48be-8312-0d268bc8d502 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:09.178923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 913/1801] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:08 2022] PUT /identity/v3/projects/50a71b32fd0d495ba9e245428f351186/users/ed37a87cea5e4714883d8b5515a4523c/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:09.178923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da85165-9492-48be-8312-0d268bc8d502 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:09.193691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b004823ea9d24e7ab4f5f94860fdd04a', 'request_id': 'req-69c8775d-abbc-43de-a678-d36f0ae15d5f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:09.324035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da85165-9492-48be-8312-0d268bc8d502 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:09.328822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd52df-e0e0-4269-81fd-8729f65f1e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:09.329461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd52df-e0e0-4269-81fd-8729f65f1e28 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:09.329871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd52df-e0e0-4269-81fd-8729f65f1e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:09.330148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd52df-e0e0-4269-81fd-8729f65f1e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:09.461821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b004823ea9d24e7ab4f5f94860fdd04a', 'request_id': 'req-69c8775d-abbc-43de-a678-d36f0ae15d5f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:09.472974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b004823ea9d24e7ab4f5f94860fdd04a', 'request_id': 'req-69c8775d-abbc-43de-a678-d36f0ae15d5f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:09.498161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd52df-e0e0-4269-81fd-8729f65f1e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:09.500286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd52df-e0e0-4269-81fd-8729f65f1e28 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:09.513054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] The token cache is being invalidate because project b004823ea9d24e7ab4f5f94860fdd04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:09.513725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:09.514007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 914/1802] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:09.515139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd52df-e0e0-4269-81fd-8729f65f1e28 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:09.516218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:09.524171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 889/1803] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:08 2022] DELETE /identity/v3/projects/b004823ea9d24e7ab4f5f94860fdd04a => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:09.527716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:09.530876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:09.534786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd52df-e0e0-4269-81fd-8729f65f1e28 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:09.566575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:09.570702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:09.570702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:09.570702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] PATH_INFO: `/v3/projects/64fc374646b54d7baa9cd77170223cca/users/ce72a82d8ed34f4a962be77d4a5c8455/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:09.595716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64fc374646b54d7baa9cd77170223cca, user_id=ce72a82d8ed34f4a962be77d4a5c8455, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:09.601766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:09.608850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:09.716625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 915/1804] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:09 2022] PUT /identity/v3/projects/64fc374646b54d7baa9cd77170223cca/users/ce72a82d8ed34f4a962be77d4a5c8455/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:09.718380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:09.724193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:09.741341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8775d-abbc-43de-a678-d36f0ae15d5f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:09.811174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:09.924269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39a922-4c0b-4907-9613-cebc7ebf8b1d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:09.949579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:09.963111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:09.963111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:09.963111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] PATH_INFO: `/v3/users/9f03ab4adbba48eb9ac3d8540a7025b8` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:09.963111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f03ab4adbba48eb9ac3d8540a7025b8)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:09.981623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:09.988393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f87e1b-447d-478b-baca-3016b1127b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:09.989304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f87e1b-447d-478b-baca-3016b1127b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:09.990609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f87e1b-447d-478b-baca-3016b1127b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:09.991198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f87e1b-447d-478b-baca-3016b1127b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:09.998288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f87e1b-447d-478b-baca-3016b1127b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:09.998288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f87e1b-447d-478b-baca-3016b1127b36 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:10.014319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 890/1805] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:10.016897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1b3b8-adc8-4579-8a26-68126eb71a6b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:10.017394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1b3b8-adc8-4579-8a26-68126eb71a6b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:10.021600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1b3b8-adc8-4579-8a26-68126eb71a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:10.100131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6db1b3b8-adc8-4579-8a26-68126eb71a6b None None] MFA Rules not processed for user `ce72a82d8ed34f4a962be77d4a5c8455`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:10.104080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6db1b3b8-adc8-4579-8a26-68126eb71a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:10.252194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 891/1806] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:10.252194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1b3b8-adc8-4579-8a26-68126eb71a6b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:10.279872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1b3b8-adc8-4579-8a26-68126eb71a6b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:10.285332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:10.286285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:10.287355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:10.288089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] PATH_INFO: `/v3/projects/50a71b32fd0d495ba9e245428f351186/users/ed37a87cea5e4714883d8b5515a4523c/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:10.291146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a71b32fd0d495ba9e245428f351186, user_id=ed37a87cea5e4714883d8b5515a4523c, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:10.293954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:10.318783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 892/1807] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:10 2022] PUT /identity/v3/projects/50a71b32fd0d495ba9e245428f351186/users/ed37a87cea5e4714883d8b5515a4523c/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:10.320073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:10.322546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:10.353956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:10.379127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f03ab4adbba48eb9ac3d8540a7025b8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:10.414258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f03ab4adbba48eb9ac3d8540a7025b8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:10.418360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa742a81-d74f-4ec5-b839-30ca86cf6c7d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:10.434307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:10.435881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:10.436610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:10.436887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:10.438629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:10.440385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f03ab4adbba48eb9ac3d8540a7025b8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:10.442445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f03ab4adbba48eb9ac3d8540a7025b8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:10.443762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0d25114-0fbb-40d9-8d3a-f23543d09711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f03ab4adbba48eb9ac3d8540a7025b8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:10.445100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:10.767842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 916/1808] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:09 2022] DELETE /identity/v3/users/9f03ab4adbba48eb9ac3d8540a7025b8 => generated 0 bytes in 1044 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:10.767842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478655dd-b3d0-4016-91a4-29e64b836153 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:10.767842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478655dd-b3d0-4016-91a4-29e64b836153 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:10.767842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478655dd-b3d0-4016-91a4-29e64b836153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:10.813149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 893/1809] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:10 2022] POST /identity/v3/projects => generated 357 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:10.813891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-478655dd-b3d0-4016-91a4-29e64b836153 None None] MFA Rules not processed for user `ed37a87cea5e4714883d8b5515a4523c`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:10.815605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-478655dd-b3d0-4016-91a4-29e64b836153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:10.816755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:10.819127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:10.895758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:10.950200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef2f15-c799-4199-8d27-d28dfef20d84 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:10.968276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 917/1810] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:10.968276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a0b85-84fa-4f8c-a8c8-7aad4b7a31e9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:10.968276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a0b85-84fa-4f8c-a8c8-7aad4b7a31e9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:10.968276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a0b85-84fa-4f8c-a8c8-7aad4b7a31e9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:10.968276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 918/1811] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:22:10.970253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a14c2-a8bb-48e9-9502-c1084381fc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:10.971194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a14c2-a8bb-48e9-9502-c1084381fc7f admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:10.971677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a14c2-a8bb-48e9-9502-c1084381fc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:10.972676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a14c2-a8bb-48e9-9502-c1084381fc7f admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:10.974320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a14c2-a8bb-48e9-9502-c1084381fc7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:10.977627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a14c2-a8bb-48e9-9502-c1084381fc7f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:10.985030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a0b85-84fa-4f8c-a8c8-7aad4b7a31e9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:10.986798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fb0a0b85-84fa-4f8c-a8c8-7aad4b7a31e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:11.096677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 894/1812] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:10 2022] POST /identity/v3/users => generated 285 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:11.098523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6e217-7862-43bc-bee0-8dadeecd0589 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:11.098523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6e217-7862-43bc-bee0-8dadeecd0589 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:11.098523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6e217-7862-43bc-bee0-8dadeecd0589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:11.110831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a0b85-84fa-4f8c-a8c8-7aad4b7a31e9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:11.127797 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e79035-7652-4b24-b50a-6366e24df8b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:11.144553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e79035-7652-4b24-b50a-6366e24df8b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:11.144978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e79035-7652-4b24-b50a-6366e24df8b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:11.145267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e79035-7652-4b24-b50a-6366e24df8b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:11.164350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-02c6e217-7862-43bc-bee0-8dadeecd0589 None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:11.169549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-02c6e217-7862-43bc-bee0-8dadeecd0589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:11.172166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e79035-7652-4b24-b50a-6366e24df8b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:11.173620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e79035-7652-4b24-b50a-6366e24df8b3 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:11.211796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 919/1813] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:11.213588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec66567-a8d2-44d7-ab9d-9aa27193ae6b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:11.214159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec66567-a8d2-44d7-ab9d-9aa27193ae6b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:11.214783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec66567-a8d2-44d7-ab9d-9aa27193ae6b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:11.219512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 920/1814] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:22:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:22:11.221045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec66567-a8d2-44d7-ab9d-9aa27193ae6b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:11.272053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec66567-a8d2-44d7-ab9d-9aa27193ae6b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:11.276449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 895/1815] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:11.276449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6e217-7862-43bc-bee0-8dadeecd0589 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:11.277407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-02c6e217-7862-43bc-bee0-8dadeecd0589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:11.285458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffa3f4-7c8b-4ebc-8fa3-a9397ed5fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:11.286168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffa3f4-7c8b-4ebc-8fa3-a9397ed5fcfa admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:11.286449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffa3f4-7c8b-4ebc-8fa3-a9397ed5fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:11.287986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffa3f4-7c8b-4ebc-8fa3-a9397ed5fcfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:11.288963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffa3f4-7c8b-4ebc-8fa3-a9397ed5fcfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:11.290447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffa3f4-7c8b-4ebc-8fa3-a9397ed5fcfa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:11.315116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 921/1816] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:11.316425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6e217-7862-43bc-bee0-8dadeecd0589 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:11.316562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bb8ef-b9cd-4a24-90a7-39b39983b6ab None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:11.316847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bb8ef-b9cd-4a24-90a7-39b39983b6ab None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:11.317106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bb8ef-b9cd-4a24-90a7-39b39983b6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:11.330814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:11.334614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:11.334614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:11.334614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] PATH_INFO: `/v3/projects/036a12fe0a6540fbaa3067164720e8a9` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:11.337358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=036a12fe0a6540fbaa3067164720e8a9)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:11.345884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:11.397874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:11.417393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-415bb8ef-b9cd-4a24-90a7-39b39983b6ab None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:11.421950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-415bb8ef-b9cd-4a24-90a7-39b39983b6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:11.449918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 922/1817] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:11.451967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bb8ef-b9cd-4a24-90a7-39b39983b6ab None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:11.453987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-415bb8ef-b9cd-4a24-90a7-39b39983b6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:11.469852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bb8ef-b9cd-4a24-90a7-39b39983b6ab None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:11.486084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012f558-26ab-41d8-a4da-e6e18177a3d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:11.486806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012f558-26ab-41d8-a4da-e6e18177a3d7 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:11.486972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012f558-26ab-41d8-a4da-e6e18177a3d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:11.487269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012f558-26ab-41d8-a4da-e6e18177a3d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:11.523486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012f558-26ab-41d8-a4da-e6e18177a3d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:11.525421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012f558-26ab-41d8-a4da-e6e18177a3d7 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:11.536136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:11.541596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 923/1818] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:11.542638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012f558-26ab-41d8-a4da-e6e18177a3d7 service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:11.561127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012f558-26ab-41d8-a4da-e6e18177a3d7 service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:11.593039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:11.605327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:11.607515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:11.608534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] PATH_INFO: `/v3/projects/eae48b61bbe148a4abcebf212b18d898/users/4d76c705da3b4c539c187ab2c45dff22/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:11.621162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036a12fe0a6540fbaa3067164720e8a9', 'request_id': 'req-d19e6282-d41e-4344-9e1e-b14fcbbae794'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:11.654032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae48b61bbe148a4abcebf212b18d898, user_id=4d76c705da3b4c539c187ab2c45dff22, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:11.657652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:11.696623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036a12fe0a6540fbaa3067164720e8a9', 'request_id': 'req-d19e6282-d41e-4344-9e1e-b14fcbbae794'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:11.705216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036a12fe0a6540fbaa3067164720e8a9', 'request_id': 'req-d19e6282-d41e-4344-9e1e-b14fcbbae794'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:11.728019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] The token cache is being invalidate because project 036a12fe0a6540fbaa3067164720e8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:11.735202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:11.735202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:11.735202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 896/1819] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:11 2022] DELETE /identity/v3/projects/036a12fe0a6540fbaa3067164720e8a9 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:11.746451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:11.754631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:11.799838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 924/1820] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:11 2022] PUT /identity/v3/projects/eae48b61bbe148a4abcebf212b18d898/users/4d76c705da3b4c539c187ab2c45dff22/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:11.799838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:11.803141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:11.905072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e6282-d41e-4344-9e1e-b14fcbbae794 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.004216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea398c-892c-4c0c-8178-f991a7528107 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.027207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:12.027988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.028342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.028638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.028858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444fbd4-dbba-4cb8-82bb-60b34b1d8e54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:12.030251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444fbd4-dbba-4cb8-82bb-60b34b1d8e54 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.030615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444fbd4-dbba-4cb8-82bb-60b34b1d8e54 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.031180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444fbd4-dbba-4cb8-82bb-60b34b1d8e54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.035744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2444fbd4-dbba-4cb8-82bb-60b34b1d8e54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:12.039223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:12.083465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444fbd4-dbba-4cb8-82bb-60b34b1d8e54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:12.089089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444fbd4-dbba-4cb8-82bb-60b34b1d8e54 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:12.122630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 925/1821] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:12.126722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b61c42-33e7-443d-8be5-3efc81cda119 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.127029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b61c42-33e7-443d-8be5-3efc81cda119 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.127925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b61c42-33e7-443d-8be5-3efc81cda119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.171621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a1b61c42-33e7-443d-8be5-3efc81cda119 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:12.186925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a1b61c42-33e7-443d-8be5-3efc81cda119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:12.250074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:12.305871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:12.308607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:12.339128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 897/1822] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:12.341268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:12.346223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:12.364311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 926/1823] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:12.375330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132445-e487-4d62-92d3-6387509c3eb5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.376450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132445-e487-4d62-92d3-6387509c3eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.376450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132445-e487-4d62-92d3-6387509c3eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.410215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-78132445-e487-4d62-92d3-6387509c3eb5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:12.412208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-78132445-e487-4d62-92d3-6387509c3eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:12.429543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 927/1824] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:12.433724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78132445-e487-4d62-92d3-6387509c3eb5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:12.444991 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78132445-e487-4d62-92d3-6387509c3eb5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.452158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:12.453039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.453486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.454024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.455094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:12.457009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:12.457506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:12.503723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 928/1825] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:22:12.506344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:12.518304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6470aa-2f35-42cf-af53-f60b9c5c4286 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.532388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12380564-034f-40cf-a053-70a2e0c578f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:12.532388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12380564-034f-40cf-a053-70a2e0c578f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.532388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12380564-034f-40cf-a053-70a2e0c578f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.532388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12380564-034f-40cf-a053-70a2e0c578f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.532896 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12380564-034f-40cf-a053-70a2e0c578f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:12.533416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a31a8c-8b58-4fd2-bc3c-46e6225cb535 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.540113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12380564-034f-40cf-a053-70a2e0c578f9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:12.550243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4857ec-b705-4cc2-9254-4df8d58af940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:12.550962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4857ec-b705-4cc2-9254-4df8d58af940 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.551273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4857ec-b705-4cc2-9254-4df8d58af940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.551881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4857ec-b705-4cc2-9254-4df8d58af940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.552756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4857ec-b705-4cc2-9254-4df8d58af940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:12.555913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4857ec-b705-4cc2-9254-4df8d58af940 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:12.567085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 898/1826] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:12.571542 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12380564-034f-40cf-a053-70a2e0c578f9 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:12.601743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12380564-034f-40cf-a053-70a2e0c578f9 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.617128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95c707-2d33-4458-9124-9364cb6222c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:12.617819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95c707-2d33-4458-9124-9364cb6222c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.618100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95c707-2d33-4458-9124-9364cb6222c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.618375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95c707-2d33-4458-9124-9364cb6222c5 admin admin] PATH_INFO: `/v3/projects/eae48b61bbe148a4abcebf212b18d898/users/4d76c705da3b4c539c187ab2c45dff22/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.627788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95c707-2d33-4458-9124-9364cb6222c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae48b61bbe148a4abcebf212b18d898, user_id=4d76c705da3b4c539c187ab2c45dff22, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:12.630975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95c707-2d33-4458-9124-9364cb6222c5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:12.663374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 899/1827] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:12 2022] PUT /identity/v3/projects/eae48b61bbe148a4abcebf212b18d898/users/4d76c705da3b4c539c187ab2c45dff22/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:12.674177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01fea-e3af-4f6f-96b8-03b491b2e939 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.674467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01fea-e3af-4f6f-96b8-03b491b2e939 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.674747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01fea-e3af-4f6f-96b8-03b491b2e939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.691543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 929/1828] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:12 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:12.694784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4857ec-b705-4cc2-9254-4df8d58af940 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:12.727812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2fe01fea-e3af-4f6f-96b8-03b491b2e939 None None] MFA Rules not processed for user `4d76c705da3b4c539c187ab2c45dff22`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:12.730734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2fe01fea-e3af-4f6f-96b8-03b491b2e939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:12.826535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4857ec-b705-4cc2-9254-4df8d58af940 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.836256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:12.837432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:12.837795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:12.838142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:12.839294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:12.842831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:12.871986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 900/1829] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:12.893493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe01fea-e3af-4f6f-96b8-03b491b2e939 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:12.897868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2fe01fea-e3af-4f6f-96b8-03b491b2e939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:12.919392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:12.965634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 930/1830] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:12 2022] POST /identity/v3/users => generated 283 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:12.967125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:12.971543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:12.983444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe01fea-e3af-4f6f-96b8-03b491b2e939 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.992748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f391ed-8185-4737-93f4-c01f905768db admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:12.994505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c344c33-c29f-4b28-8955-25c0e55b8b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:12.995339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c344c33-c29f-4b28-8955-25c0e55b8b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.001358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c344c33-c29f-4b28-8955-25c0e55b8b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.001358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c344c33-c29f-4b28-8955-25c0e55b8b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.001553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.001553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.001553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.001874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] PATH_INFO: `/v3/projects/58ffbd1872564385b10c359289567ecc` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.003046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=58ffbd1872564385b10c359289567ecc)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.005034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.011450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c344c33-c29f-4b28-8955-25c0e55b8b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.013274 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c344c33-c29f-4b28-8955-25c0e55b8b30 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.027461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 901/1831] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:13.030024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c344c33-c29f-4b28-8955-25c0e55b8b30 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.041674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c344c33-c29f-4b28-8955-25c0e55b8b30 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.046833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370f88a-f955-4b4c-8589-75a42cef9579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.047605 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370f88a-f955-4b4c-8589-75a42cef9579 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.048504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370f88a-f955-4b4c-8589-75a42cef9579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.048504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370f88a-f955-4b4c-8589-75a42cef9579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.048798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370f88a-f955-4b4c-8589-75a42cef9579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.050595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370f88a-f955-4b4c-8589-75a42cef9579 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.062912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 902/1832] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:13.069299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370f88a-f955-4b4c-8589-75a42cef9579 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.085407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370f88a-f955-4b4c-8589-75a42cef9579 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.086189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:13.103045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a125c6-a433-408f-9a31-374b58ea5e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.103669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a125c6-a433-408f-9a31-374b58ea5e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.103933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a125c6-a433-408f-9a31-374b58ea5e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.104202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a125c6-a433-408f-9a31-374b58ea5e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.104850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a125c6-a433-408f-9a31-374b58ea5e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.106601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a125c6-a433-408f-9a31-374b58ea5e59 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.119039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 903/1833] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:13.131702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a125c6-a433-408f-9a31-374b58ea5e59 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.144777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58ffbd1872564385b10c359289567ecc', 'request_id': 'req-b54e159c-3393-40f1-a070-97b54546faaf'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:13.148029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a125c6-a433-408f-9a31-374b58ea5e59 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.224776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba718dd-e2c6-47bf-bf25-65af69fbe5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.225884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba718dd-e2c6-47bf-bf25-65af69fbe5c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.226104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba718dd-e2c6-47bf-bf25-65af69fbe5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.226383 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba718dd-e2c6-47bf-bf25-65af69fbe5c7 admin admin] PATH_INFO: `/v3/projects/d85432199e104fe0ab8905290aa6c0c9/users/e9bc701d922249769b3aa13e5444a8a4/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.246779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba718dd-e2c6-47bf-bf25-65af69fbe5c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d85432199e104fe0ab8905290aa6c0c9, user_id=e9bc701d922249769b3aa13e5444a8a4, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.249524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba718dd-e2c6-47bf-bf25-65af69fbe5c7 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.260636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58ffbd1872564385b10c359289567ecc', 'request_id': 'req-b54e159c-3393-40f1-a070-97b54546faaf'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:13.267127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58ffbd1872564385b10c359289567ecc', 'request_id': 'req-b54e159c-3393-40f1-a070-97b54546faaf'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:13.280054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] The token cache is being invalidate because project 58ffbd1872564385b10c359289567ecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:13.280476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:13.281154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b54e159c-3393-40f1-a070-97b54546faaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:13.282889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 931/1834] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:12 2022] DELETE /identity/v3/projects/58ffbd1872564385b10c359289567ecc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:13.288538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95aa80-25a9-4bf8-a9e3-a5c6e60e18a1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.288829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95aa80-25a9-4bf8-a9e3-a5c6e60e18a1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.289110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95aa80-25a9-4bf8-a9e3-a5c6e60e18a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.314917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7b95aa80-25a9-4bf8-a9e3-a5c6e60e18a1 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:13.316550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7b95aa80-25a9-4bf8-a9e3-a5c6e60e18a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:13.326539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 932/1835] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:13.327495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95aa80-25a9-4bf8-a9e3-a5c6e60e18a1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.328872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7b95aa80-25a9-4bf8-a9e3-a5c6e60e18a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:13.362141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 904/1836] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:13 2022] PUT /identity/v3/projects/d85432199e104fe0ab8905290aa6c0c9/users/e9bc701d922249769b3aa13e5444a8a4/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:13.364971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5ed58-5bfb-4f18-8cee-b7a5eff7abc0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.364971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5ed58-5bfb-4f18-8cee-b7a5eff7abc0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.368882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5ed58-5bfb-4f18-8cee-b7a5eff7abc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.389176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-dcf5ed58-5bfb-4f18-8cee-b7a5eff7abc0 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:13.390623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-dcf5ed58-5bfb-4f18-8cee-b7a5eff7abc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:13.401817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95aa80-25a9-4bf8-a9e3-a5c6e60e18a1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.467825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.467825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.467825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.467825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.468480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:13.492156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 905/1837] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:13.496521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b28ea-a26c-4a2f-bd63-18299b9f04b0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.496521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b28ea-a26c-4a2f-bd63-18299b9f04b0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.496521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b28ea-a26c-4a2f-bd63-18299b9f04b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.529968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.531519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.538325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-9b2b28ea-a26c-4a2f-bd63-18299b9f04b0 None None] MFA Rules not processed for user `e9bc701d922249769b3aa13e5444a8a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:13.539728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9b2b28ea-a26c-4a2f-bd63-18299b9f04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:13.545521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 933/1838] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:13.548481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.562252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be971c8-08a1-4631-801e-e0853ed9bc3e service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.574982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.575625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.575905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.576251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.576815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:13.577113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.578761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.593893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 934/1839] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:13.598301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.609850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3733c5-8546-4af9-bdeb-daaad7b44d72 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.616445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c90a2-44a7-4d5f-a040-d780cf934ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.617460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c90a2-44a7-4d5f-a040-d780cf934ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.617604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c90a2-44a7-4d5f-a040-d780cf934ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.617804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c90a2-44a7-4d5f-a040-d780cf934ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.618452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c90a2-44a7-4d5f-a040-d780cf934ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.620526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c90a2-44a7-4d5f-a040-d780cf934ebb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.660637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 906/1840] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:13.665801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b28ea-a26c-4a2f-bd63-18299b9f04b0 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.667338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9b2b28ea-a26c-4a2f-bd63-18299b9f04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:13.682669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b28ea-a26c-4a2f-bd63-18299b9f04b0 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.687324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd93d7-47df-46a6-a512-a354853bfb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.687893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd93d7-47df-46a6-a512-a354853bfb5e admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.688172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd93d7-47df-46a6-a512-a354853bfb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.688446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd93d7-47df-46a6-a512-a354853bfb5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.689070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd93d7-47df-46a6-a512-a354853bfb5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.690964 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd93d7-47df-46a6-a512-a354853bfb5e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.730683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 935/1841] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:13 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:13.734030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c90a2-44a7-4d5f-a040-d780cf934ebb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.799457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c90a2-44a7-4d5f-a040-d780cf934ebb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.805194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0559b-7265-4d1a-8a70-2a8ac6843ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.806146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0559b-7265-4d1a-8a70-2a8ac6843ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.806430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0559b-7265-4d1a-8a70-2a8ac6843ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.806709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0559b-7265-4d1a-8a70-2a8ac6843ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.812363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0559b-7265-4d1a-8a70-2a8ac6843ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.812363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0559b-7265-4d1a-8a70-2a8ac6843ad2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.812543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 907/1842] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:13 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:13.815853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd93d7-47df-46a6-a512-a354853bfb5e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.884410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd93d7-47df-46a6-a512-a354853bfb5e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.889200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1c711-2c54-4f38-a442-564d0b09e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.889801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1c711-2c54-4f38-a442-564d0b09e2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.890083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1c711-2c54-4f38-a442-564d0b09e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.890356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1c711-2c54-4f38-a442-564d0b09e2ce admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.890973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1c711-2c54-4f38-a442-564d0b09e2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.894550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1c711-2c54-4f38-a442-564d0b09e2ce admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.937707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 936/1843] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:13 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:13.941615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0559b-7265-4d1a-8a70-2a8ac6843ad2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.952896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0559b-7265-4d1a-8a70-2a8ac6843ad2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:13.958234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62dccc8-72ca-4642-8572-2eb6a2f9fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:13.958601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dccc8-72ca-4642-8572-2eb6a2f9fa48 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:13.959035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dccc8-72ca-4642-8572-2eb6a2f9fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:13.959400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dccc8-72ca-4642-8572-2eb6a2f9fa48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:13.960051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62dccc8-72ca-4642-8572-2eb6a2f9fa48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:13.961649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62dccc8-72ca-4642-8572-2eb6a2f9fa48 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:13.971937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 937/1844] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:13.976592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62dccc8-72ca-4642-8572-2eb6a2f9fa48 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:13.986360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62dccc8-72ca-4642-8572-2eb6a2f9fa48 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.000364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc626f56-1d5d-4d27-b550-838bfc97ba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.000364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc626f56-1d5d-4d27-b550-838bfc97ba6d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.000364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc626f56-1d5d-4d27-b550-838bfc97ba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.000364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc626f56-1d5d-4d27-b550-838bfc97ba6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.000699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc626f56-1d5d-4d27-b550-838bfc97ba6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.000699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc626f56-1d5d-4d27-b550-838bfc97ba6d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.007586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 938/1845] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:14.012159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc626f56-1d5d-4d27-b550-838bfc97ba6d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.024746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc626f56-1d5d-4d27-b550-838bfc97ba6d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.028417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 908/1846] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:13 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:14.037648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b571a-5822-4a98-886e-2a56772a0167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.037648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b571a-5822-4a98-886e-2a56772a0167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.037648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b571a-5822-4a98-886e-2a56772a0167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.037648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b571a-5822-4a98-886e-2a56772a0167 admin admin] PATH_INFO: `/v3/projects/7579601d618243d1bb268b2e0c7864d9/users/e8891669ad0b41738b95c47d05739539/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.040800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1c711-2c54-4f38-a442-564d0b09e2ce admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.049333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b571a-5822-4a98-886e-2a56772a0167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7579601d618243d1bb268b2e0c7864d9, user_id=e8891669ad0b41738b95c47d05739539, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.051111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1c711-2c54-4f38-a442-564d0b09e2ce admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.052282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b571a-5822-4a98-886e-2a56772a0167 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.057729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:14.058325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30556b1e-aa56-42cd-a0c6-bb3406f66a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.060709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556b1e-aa56-42cd-a0c6-bb3406f66a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.060709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556b1e-aa56-42cd-a0c6-bb3406f66a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.060709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556b1e-aa56-42cd-a0c6-bb3406f66a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.060709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30556b1e-aa56-42cd-a0c6-bb3406f66a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.061810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30556b1e-aa56-42cd-a0c6-bb3406f66a91 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.075988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 909/1847] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:14.076857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30556b1e-aa56-42cd-a0c6-bb3406f66a91 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.087551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30556b1e-aa56-42cd-a0c6-bb3406f66a91 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.093042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7001c84-8429-4310-a0b3-ad56c63bd793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.093209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7001c84-8429-4310-a0b3-ad56c63bd793 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.093494 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7001c84-8429-4310-a0b3-ad56c63bd793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.093771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7001c84-8429-4310-a0b3-ad56c63bd793 admin admin] PATH_INFO: `/v3/projects/aad1c8ee180045a3808dc38e136fd845/users/8a947c8e40884778b34c0f0a9e206405/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.114788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7001c84-8429-4310-a0b3-ad56c63bd793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad1c8ee180045a3808dc38e136fd845, user_id=8a947c8e40884778b34c0f0a9e206405, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.117931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7001c84-8429-4310-a0b3-ad56c63bd793 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.141395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 939/1848] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:14 2022] PUT /identity/v3/projects/7579601d618243d1bb268b2e0c7864d9/users/e8891669ad0b41738b95c47d05739539/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:14.152047 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc2c71-3179-489d-b315-96cfe2ae6a00 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.153062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc2c71-3179-489d-b315-96cfe2ae6a00 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.153062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc2c71-3179-489d-b315-96cfe2ae6a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.181461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c8fc2c71-3179-489d-b315-96cfe2ae6a00 None None] MFA Rules not processed for user `e8891669ad0b41738b95c47d05739539`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:14.182690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c8fc2c71-3179-489d-b315-96cfe2ae6a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:14.187890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 910/1849] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:14 2022] PUT /identity/v3/projects/aad1c8ee180045a3808dc38e136fd845/users/8a947c8e40884778b34c0f0a9e206405/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:14.191753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7001c84-8429-4310-a0b3-ad56c63bd793 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.252627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 940/1850] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:14.257867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc2c71-3179-489d-b315-96cfe2ae6a00 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.259503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7001c84-8429-4310-a0b3-ad56c63bd793 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.264570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327808c-ff88-4345-be35-c0539b42fe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.265176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327808c-ff88-4345-be35-c0539b42fe76 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.265454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327808c-ff88-4345-be35-c0539b42fe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.265998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327808c-ff88-4345-be35-c0539b42fe76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.266954 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327808c-ff88-4345-be35-c0539b42fe76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.277591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:14.277731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc2c71-3179-489d-b315-96cfe2ae6a00 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.277830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327808c-ff88-4345-be35-c0539b42fe76 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.277919 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985a6ea-70fd-43d7-901b-e1ef10d93a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.278050 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a6ea-70fd-43d7-901b-e1ef10d93a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.278324 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a6ea-70fd-43d7-901b-e1ef10d93a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.278603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a6ea-70fd-43d7-901b-e1ef10d93a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.279289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985a6ea-70fd-43d7-901b-e1ef10d93a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.281227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985a6ea-70fd-43d7-901b-e1ef10d93a49 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.283036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 911/1851] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:14.285921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327808c-ff88-4345-be35-c0539b42fe76 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.297364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327808c-ff88-4345-be35-c0539b42fe76 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.304750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114ccdc-8d78-400f-9749-be65bcc80b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.305767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ccdc-8d78-400f-9749-be65bcc80b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.306111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ccdc-8d78-400f-9749-be65bcc80b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.306390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ccdc-8d78-400f-9749-be65bcc80b78 admin admin] PATH_INFO: `/v3/projects/aad1c8ee180045a3808dc38e136fd845/users/8a947c8e40884778b34c0f0a9e206405/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.308420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114ccdc-8d78-400f-9749-be65bcc80b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad1c8ee180045a3808dc38e136fd845, user_id=8a947c8e40884778b34c0f0a9e206405, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.311673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114ccdc-8d78-400f-9749-be65bcc80b78 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.322788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 912/1852] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:14 2022] PUT /identity/v3/projects/aad1c8ee180045a3808dc38e136fd845/users/8a947c8e40884778b34c0f0a9e206405/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:14.334556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750a61e-f73e-4257-ae56-8fee72255a93 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.334890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750a61e-f73e-4257-ae56-8fee72255a93 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.335183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750a61e-f73e-4257-ae56-8fee72255a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.366237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8750a61e-f73e-4257-ae56-8fee72255a93 None None] MFA Rules not processed for user `8a947c8e40884778b34c0f0a9e206405`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:14.367528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8750a61e-f73e-4257-ae56-8fee72255a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:14.386538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 941/1853] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:14 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:14.391827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985a6ea-70fd-43d7-901b-e1ef10d93a49 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.442583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 913/1854] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:14.456418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750a61e-f73e-4257-ae56-8fee72255a93 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.457980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8750a61e-f73e-4257-ae56-8fee72255a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:14.473435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985a6ea-70fd-43d7-901b-e1ef10d93a49 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.478885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341064-d24f-42ee-9e3e-867e1a419536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.479616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341064-d24f-42ee-9e3e-867e1a419536 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.479900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341064-d24f-42ee-9e3e-867e1a419536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.480176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341064-d24f-42ee-9e3e-867e1a419536 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.480877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb341064-d24f-42ee-9e3e-867e1a419536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.482871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb341064-d24f-42ee-9e3e-867e1a419536 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.523425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750a61e-f73e-4257-ae56-8fee72255a93 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.580189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9777-91cd-4b10-9f5d-97310c58f85e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.581065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9777-91cd-4b10-9f5d-97310c58f85e service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.581472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9777-91cd-4b10-9f5d-97310c58f85e service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.581986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9777-91cd-4b10-9f5d-97310c58f85e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.618256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 942/1855] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:14 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:14.622130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341064-d24f-42ee-9e3e-867e1a419536 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.632666 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341064-d24f-42ee-9e3e-867e1a419536 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.638618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17876e33-d4ec-47d4-94d9-69f92e4ffc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.639517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17876e33-d4ec-47d4-94d9-69f92e4ffc93 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.639939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17876e33-d4ec-47d4-94d9-69f92e4ffc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.640352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17876e33-d4ec-47d4-94d9-69f92e4ffc93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.641297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17876e33-d4ec-47d4-94d9-69f92e4ffc93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.643571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17876e33-d4ec-47d4-94d9-69f92e4ffc93 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.655167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 943/1856] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:14.658850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17876e33-d4ec-47d4-94d9-69f92e4ffc93 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.670686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17876e33-d4ec-47d4-94d9-69f92e4ffc93 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.676496 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de384b0-f254-41b9-b015-2a1076cd7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.677268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de384b0-f254-41b9-b015-2a1076cd7e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.679029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de384b0-f254-41b9-b015-2a1076cd7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.679029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de384b0-f254-41b9-b015-2a1076cd7e41 admin admin] PATH_INFO: `/v3/projects/372f833c66d849a389672f036a2da1a7/users/898fc0e67b134f858e6ba677542d4626/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.681279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c9777-91cd-4b10-9f5d-97310c58f85e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.682957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c9777-91cd-4b10-9f5d-97310c58f85e service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.695559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 914/1857] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:14.697143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de384b0-f254-41b9-b015-2a1076cd7e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372f833c66d849a389672f036a2da1a7, user_id=898fc0e67b134f858e6ba677542d4626, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.699900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de384b0-f254-41b9-b015-2a1076cd7e41 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.730658 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:14.793792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 944/1858] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:14 2022] PUT /identity/v3/projects/372f833c66d849a389672f036a2da1a7/users/898fc0e67b134f858e6ba677542d4626/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:14.797761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9777-91cd-4b10-9f5d-97310c58f85e service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.808302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9777-91cd-4b10-9f5d-97310c58f85e service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.827678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:14.886065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9225a-318d-4bb6-ac1a-5419a31402c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.886716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9225a-318d-4bb6-ac1a-5419a31402c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.887049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9225a-318d-4bb6-ac1a-5419a31402c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.887333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9225a-318d-4bb6-ac1a-5419a31402c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.888134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9225a-318d-4bb6-ac1a-5419a31402c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.890971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9225a-318d-4bb6-ac1a-5419a31402c8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.904420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 915/1859] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:14.908584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de384b0-f254-41b9-b015-2a1076cd7e41 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:14.921741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de384b0-f254-41b9-b015-2a1076cd7e41 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:14.926492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4136c3-8bad-4e89-9559-50e004495f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:14.927359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4136c3-8bad-4e89-9559-50e004495f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.927742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4136c3-8bad-4e89-9559-50e004495f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.928031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4136c3-8bad-4e89-9559-50e004495f77 admin admin] PATH_INFO: `/v3/projects/372f833c66d849a389672f036a2da1a7/users/898fc0e67b134f858e6ba677542d4626/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.933811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4136c3-8bad-4e89-9559-50e004495f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372f833c66d849a389672f036a2da1a7, user_id=898fc0e67b134f858e6ba677542d4626, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:14.936646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4136c3-8bad-4e89-9559-50e004495f77 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:14.948963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 945/1860] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:14 2022] PUT /identity/v3/projects/372f833c66d849a389672f036a2da1a7/users/898fc0e67b134f858e6ba677542d4626/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:14.959728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca74f6-47da-4e64-b78a-9c4324a6902d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:14.960039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca74f6-47da-4e64-b78a-9c4324a6902d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:14.960315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca74f6-47da-4e64-b78a-9c4324a6902d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:14.991840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-9dca74f6-47da-4e64-b78a-9c4324a6902d None None] MFA Rules not processed for user `898fc0e67b134f858e6ba677542d4626`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:14.993090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9dca74f6-47da-4e64-b78a-9c4324a6902d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:15.092735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 916/1861] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:15.102894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4136c3-8bad-4e89-9559-50e004495f77 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:15.165700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:15.230464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4136c3-8bad-4e89-9559-50e004495f77 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:15.235466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9bcd3-f5de-42a7-9393-25db6fffc4c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:15.239549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9bcd3-f5de-42a7-9393-25db6fffc4c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:15.240699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9bcd3-f5de-42a7-9393-25db6fffc4c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:15.241381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9bcd3-f5de-42a7-9393-25db6fffc4c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:15.245862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:15.256880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9bcd3-f5de-42a7-9393-25db6fffc4c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:15.259714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9bcd3-f5de-42a7-9393-25db6fffc4c8 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:15.272286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 946/1862] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:15.318955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca74f6-47da-4e64-b78a-9c4324a6902d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:15.402437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca74f6-47da-4e64-b78a-9c4324a6902d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:15.407786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:15.408508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:15.408864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:15.409257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] PATH_INFO: `/v3/users/e8891669ad0b41738b95c47d05739539` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:15.411160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8891669ad0b41738b95c47d05739539)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:15.413858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:15.595438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8891669ad0b41738b95c47d05739539'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:15.617616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:15.629567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8891669ad0b41738b95c47d05739539'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:15.640622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8891669ad0b41738b95c47d05739539'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:15.643516 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8891669ad0b41738b95c47d05739539'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:15.650415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8891669ad0b41738b95c47d05739539'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:15.670435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:15.783049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 917/1863] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:15 2022] DELETE /identity/v3/users/e8891669ad0b41738b95c47d05739539 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:15.788604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6bd3-3101-46c8-b8d3-0c6e5dc0e687 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:15.788997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6bd3-3101-46c8-b8d3-0c6e5dc0e687 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:15.789271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6bd3-3101-46c8-b8d3-0c6e5dc0e687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:15.815833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-231f6bd3-3101-46c8-b8d3-0c6e5dc0e687 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:15.817319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-231f6bd3-3101-46c8-b8d3-0c6e5dc0e687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:15.886834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 947/1864] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:16.010529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:16.012201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:16.087335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97c1d4-4d1c-4152-9a72-6a90c77ed0d8 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:16.090888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:16.091531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:16.091807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:16.092075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:16.101729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:16.103508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:16.124864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 918/1865] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:16.208424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f6bd3-3101-46c8-b8d3-0c6e5dc0e687 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:16.210782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-231f6bd3-3101-46c8-b8d3-0c6e5dc0e687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:16.223815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f6bd3-3101-46c8-b8d3-0c6e5dc0e687 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:16.233795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:16.234513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:16.234895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:16.235216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] PATH_INFO: `/v3/users/e9bc701d922249769b3aa13e5444a8a4` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:16.236486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9bc701d922249769b3aa13e5444a8a4)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:16.238448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:16.530974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:16.753155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9bc701d922249769b3aa13e5444a8a4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:16.787459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9bc701d922249769b3aa13e5444a8a4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:16.830150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9bc701d922249769b3aa13e5444a8a4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:16.837431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9bc701d922249769b3aa13e5444a8a4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:16.845045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78885f82-1841-4717-b584-14ef2fad6d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9bc701d922249769b3aa13e5444a8a4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:16.973633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 948/1866] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:16 2022] DELETE /identity/v3/users/e9bc701d922249769b3aa13e5444a8a4 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:16.977888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:16.978441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38e2ff-ecc6-4e28-9b32-5f5e5acb464f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:16.978738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38e2ff-ecc6-4e28-9b32-5f5e5acb464f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:16.979264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38e2ff-ecc6-4e28-9b32-5f5e5acb464f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:16.980101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:16.999099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4eac9-9f24-4ae5-8b0a-04e1d338d5d4 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:17.032145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-af38e2ff-ecc6-4e28-9b32-5f5e5acb464f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:17.033436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-af38e2ff-ecc6-4e28-9b32-5f5e5acb464f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:17.093771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:17.094597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:17.094878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:17.095362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] PATH_INFO: `/v3/projects/7579601d618243d1bb268b2e0c7864d9` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:17.096863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7579601d618243d1bb268b2e0c7864d9)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:17.103858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:17.174463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 949/1867] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:17.184348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38e2ff-ecc6-4e28-9b32-5f5e5acb464f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:17.186200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-af38e2ff-ecc6-4e28-9b32-5f5e5acb464f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:17.256123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38e2ff-ecc6-4e28-9b32-5f5e5acb464f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:17.285120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:17.285120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:17.285120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:17.285120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:17.285517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:17.285517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:17.289172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 950/1868] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:17.431059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7579601d618243d1bb268b2e0c7864d9', 'request_id': 'req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:17.621751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7579601d618243d1bb268b2e0c7864d9', 'request_id': 'req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:17.632807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7579601d618243d1bb268b2e0c7864d9', 'request_id': 'req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:17.652414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] The token cache is being invalidate because project 7579601d618243d1bb268b2e0c7864d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:17.652717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:17.653912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:17.655740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 919/1869] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:16 2022] DELETE /identity/v3/projects/7579601d618243d1bb268b2e0c7864d9 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:17.661013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:17.662977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:17.687314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f492bfe-8c70-48d4-937d-e4624a44c9a1 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:17.716104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:17.821559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:17.822509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:17.822940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:17.823542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] PATH_INFO: `/v3/users/898fc0e67b134f858e6ba677542d4626` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:17.825157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=898fc0e67b134f858e6ba677542d4626)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:17.827891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:18.071391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:18.073178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:18.096115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:18.105445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247b2a5-2bf1-4328-bd1c-3d1455d75e4f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:18.113656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:18.114854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:18.115527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:18.116142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] PATH_INFO: `/v3/projects/d85432199e104fe0ab8905290aa6c0c9` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:18.118146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d85432199e104fe0ab8905290aa6c0c9)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:18.118557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '898fc0e67b134f858e6ba677542d4626'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.123365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:18.171729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '898fc0e67b134f858e6ba677542d4626'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.206303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '898fc0e67b134f858e6ba677542d4626'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.208251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '898fc0e67b134f858e6ba677542d4626'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.209754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '898fc0e67b134f858e6ba677542d4626'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.437166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 951/1870] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:17 2022] DELETE /identity/v3/users/898fc0e67b134f858e6ba677542d4626 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:18.458262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd85432199e104fe0ab8905290aa6c0c9', 'request_id': 'req-2ecb161c-4210-4a4a-a58a-f50952b89f72'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.610679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd85432199e104fe0ab8905290aa6c0c9', 'request_id': 'req-2ecb161c-4210-4a4a-a58a-f50952b89f72'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.618567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd85432199e104fe0ab8905290aa6c0c9', 'request_id': 'req-2ecb161c-4210-4a4a-a58a-f50952b89f72'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.631505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] The token cache is being invalidate because project d85432199e104fe0ab8905290aa6c0c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:18.637780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.637780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:18.637780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 920/1871] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:18 2022] DELETE /identity/v3/projects/d85432199e104fe0ab8905290aa6c0c9 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:18.644382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:18.646560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:18.791478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3535e0d-abea-4320-b315-159449b0e4b3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:18.794942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:18.795700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:18.796082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:18.796430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] PATH_INFO: `/v3/users/8a947c8e40884778b34c0f0a9e206405` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:18.799768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a947c8e40884778b34c0f0a9e206405)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:18.802558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:18.961180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:19.075345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a947c8e40884778b34c0f0a9e206405'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.090224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a947c8e40884778b34c0f0a9e206405'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.124185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a947c8e40884778b34c0f0a9e206405'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.126001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a947c8e40884778b34c0f0a9e206405'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.127433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ece21394-e8a3-4b19-97d3-89bd54a8b762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a947c8e40884778b34c0f0a9e206405'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.132134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:19.134227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:19.268622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 952/1872] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:18 2022] DELETE /identity/v3/users/8a947c8e40884778b34c0f0a9e206405 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:19.291511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb161c-4210-4a4a-a58a-f50952b89f72 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:19.411690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:19.412390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:19.412684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:19.413014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] PATH_INFO: `/v3/projects/372f833c66d849a389672f036a2da1a7` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:19.414536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=372f833c66d849a389672f036a2da1a7)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:19.416282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:19.439028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:19.606218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372f833c66d849a389672f036a2da1a7', 'request_id': 'req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.673502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372f833c66d849a389672f036a2da1a7', 'request_id': 'req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.683490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372f833c66d849a389672f036a2da1a7', 'request_id': 'req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.700403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] The token cache is being invalidate because project 372f833c66d849a389672f036a2da1a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:19.700403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.700403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c79ab2c8-a5c6-455e-a39e-120fbf2a6b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:19.700403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 921/1873] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:19 2022] DELETE /identity/v3/projects/372f833c66d849a389672f036a2da1a7 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:19.705059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481777-88d4-433b-bf17-997312762f70 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:19.710494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481777-88d4-433b-bf17-997312762f70 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:19.711149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481777-88d4-433b-bf17-997312762f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:19.744601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cd481777-88d4-433b-bf17-997312762f70 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:19.746418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cd481777-88d4-433b-bf17-997312762f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:19.842173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 953/1874] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:19.848263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e9059-326c-4016-a143-057f8ad6b32b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:19.850037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e9059-326c-4016-a143-057f8ad6b32b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:19.850037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e9059-326c-4016-a143-057f8ad6b32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:19.884920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0c6e9059-326c-4016-a143-057f8ad6b32b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:19.889726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0c6e9059-326c-4016-a143-057f8ad6b32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:19.912609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd481777-88d4-433b-bf17-997312762f70 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:19.922123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 922/1875] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:19.929268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd481777-88d4-433b-bf17-997312762f70 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:19.931035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:19.931679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:19.931962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:19.932224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:19.933066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:19.933399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:19.935106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:19.956176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 954/1876] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:19.958103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e9059-326c-4016-a143-057f8ad6b32b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:19.969474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e9059-326c-4016-a143-057f8ad6b32b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:19.974725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e52703-23bb-4956-935b-c18b955fca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:20.018340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e52703-23bb-4956-935b-c18b955fca80 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.018340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e52703-23bb-4956-935b-c18b955fca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.018340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e52703-23bb-4956-935b-c18b955fca80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:20.018340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e52703-23bb-4956-935b-c18b955fca80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:20.018340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e52703-23bb-4956-935b-c18b955fca80 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:20.460555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:20.461503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 923/1877] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:19 2022] POST /identity/v3/projects => generated 381 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:20.463510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:20.465892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e52703-23bb-4956-935b-c18b955fca80 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:20.485450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d2089-e6ec-456e-bf89-463e06b25a66 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:20.537778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e52703-23bb-4956-935b-c18b955fca80 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:20.542398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:20.543019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.543304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.543572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:20.544187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:20.545351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:20.545989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.546153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:20.546285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.546550 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] PATH_INFO: `/v3/projects/aad1c8ee180045a3808dc38e136fd845` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:20.547693 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad1c8ee180045a3808dc38e136fd845)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:20.549595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:20.718921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 924/1878] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:20 2022] POST /identity/v3/users => generated 291 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:20.718921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:20.725112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:20.727570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad1c8ee180045a3808dc38e136fd845', 'request_id': 'req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:20.742880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b418-d92a-4915-af55-a7798c8be9a4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:20.792754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:20.793394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.793673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.794296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] PATH_INFO: `/v3/users/15d895ea7f484146acf9e97c749c93af` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:20.795104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d895ea7f484146acf9e97c749c93af)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:20.797024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:20.804992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad1c8ee180045a3808dc38e136fd845', 'request_id': 'req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:20.820917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad1c8ee180045a3808dc38e136fd845', 'request_id': 'req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:20.834547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] The token cache is being invalidate because project aad1c8ee180045a3808dc38e136fd845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:20.835089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:20.835714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:20.837218 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 955/1879] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:20 2022] DELETE /identity/v3/projects/aad1c8ee180045a3808dc38e136fd845 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:20.841852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:20.842684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:20.855158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fc649-ef2d-4f36-a1c9-6611e9800b3f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:20.859995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f570f2c-c195-425d-914f-4f99c018cbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:20.860607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570f2c-c195-425d-914f-4f99c018cbbf admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.860891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570f2c-c195-425d-914f-4f99c018cbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.861168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570f2c-c195-425d-914f-4f99c018cbbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:20.861824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f570f2c-c195-425d-914f-4f99c018cbbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:20.863624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f570f2c-c195-425d-914f-4f99c018cbbf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:20.875359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 956/1880] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:20.879537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a969a-7567-42e7-81f6-c605dfb03349 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.881038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a969a-7567-42e7-81f6-c605dfb03349 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.881038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a969a-7567-42e7-81f6-c605dfb03349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:20.900388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-576a969a-7567-42e7-81f6-c605dfb03349 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:20.901597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-576a969a-7567-42e7-81f6-c605dfb03349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:20.910020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 957/1881] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:20.910832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a969a-7567-42e7-81f6-c605dfb03349 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:20.920540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a969a-7567-42e7-81f6-c605dfb03349 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:20.925225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff705d-1971-4b09-972e-7e619b30d019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:20.925832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff705d-1971-4b09-972e-7e619b30d019 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.926110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff705d-1971-4b09-972e-7e619b30d019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.926380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff705d-1971-4b09-972e-7e619b30d019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:20.927572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff705d-1971-4b09-972e-7e619b30d019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:20.929143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff705d-1971-4b09-972e-7e619b30d019 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:20.939031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 958/1882] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:20.940317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a629d-a00f-4fc6-9398-c1ac44b490ef None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.940600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a629d-a00f-4fc6-9398-c1ac44b490ef None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.940874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a629d-a00f-4fc6-9398-c1ac44b490ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:20.961412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e35a629d-a00f-4fc6-9398-c1ac44b490ef None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:20.967250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e35a629d-a00f-4fc6-9398-c1ac44b490ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:20.970418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 959/1883] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:20.971336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a629d-a00f-4fc6-9398-c1ac44b490ef None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:20.980628 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a629d-a00f-4fc6-9398-c1ac44b490ef None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:20.985119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:20.985731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:20.986005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:20.986274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] PATH_INFO: `/v3/projects/ece10fe09dc449529c7e3ccf88f42e70/users/3fd2163da44f44f8931ac7fa11e11313/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.002213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece10fe09dc449529c7e3ccf88f42e70, user_id=3fd2163da44f44f8931ac7fa11e11313, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.006369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.029289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d895ea7f484146acf9e97c749c93af'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:21.041392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d895ea7f484146acf9e97c749c93af'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:21.048408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d895ea7f484146acf9e97c749c93af'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:21.049180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d895ea7f484146acf9e97c749c93af'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:21.049800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ca567cbf-75bb-47e8-ba61-9ee1b9035c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d895ea7f484146acf9e97c749c93af'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:21.079273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 960/1884] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:20 2022] PUT /identity/v3/projects/ece10fe09dc449529c7e3ccf88f42e70/users/3fd2163da44f44f8931ac7fa11e11313/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:21.080026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.081664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:21.148619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93a9e2-d17c-44e8-8d0f-61980a7612a2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.153233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd314aac-e257-4f75-8fa4-4c0e6a61bada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.153827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd314aac-e257-4f75-8fa4-4c0e6a61bada admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.154104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd314aac-e257-4f75-8fa4-4c0e6a61bada admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.154374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd314aac-e257-4f75-8fa4-4c0e6a61bada admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.155020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd314aac-e257-4f75-8fa4-4c0e6a61bada admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:21.157507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd314aac-e257-4f75-8fa4-4c0e6a61bada admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.157507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd314aac-e257-4f75-8fa4-4c0e6a61bada admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.165455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 925/1885] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:20 2022] DELETE /identity/v3/users/15d895ea7f484146acf9e97c749c93af => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:21.168273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1a15e-9acf-4855-93bc-aeb32d97af4b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.168703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1a15e-9acf-4855-93bc-aeb32d97af4b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.169078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1a15e-9acf-4855-93bc-aeb32d97af4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.169802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 961/1886] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:21.170990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2984cf3-1e85-4b4e-a76b-9f0cd60c412a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.171373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2984cf3-1e85-4b4e-a76b-9f0cd60c412a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.171661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2984cf3-1e85-4b4e-a76b-9f0cd60c412a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.193982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d2984cf3-1e85-4b4e-a76b-9f0cd60c412a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:21.195235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d2984cf3-1e85-4b4e-a76b-9f0cd60c412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:21.209395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-04c1a15e-9acf-4855-93bc-aeb32d97af4b None None] MFA Rules not processed for user `3fd2163da44f44f8931ac7fa11e11313`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:21.210798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-04c1a15e-9acf-4855-93bc-aeb32d97af4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:21.229356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:21.290695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 962/1887] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:21.290695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2984cf3-1e85-4b4e-a76b-9f0cd60c412a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.295182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 926/1888] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:21.296103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1a15e-9acf-4855-93bc-aeb32d97af4b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.297475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-04c1a15e-9acf-4855-93bc-aeb32d97af4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:21.313448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2984cf3-1e85-4b4e-a76b-9f0cd60c412a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.313816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1a15e-9acf-4855-93bc-aeb32d97af4b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.321653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.321877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.322045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.322318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.323028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.324960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.326795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972986c2-52ef-4d3e-b106-007dce3ee4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.327842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972986c2-52ef-4d3e-b106-007dce3ee4ca admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.327967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972986c2-52ef-4d3e-b106-007dce3ee4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.332084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972986c2-52ef-4d3e-b106-007dce3ee4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.332084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972986c2-52ef-4d3e-b106-007dce3ee4ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.332084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972986c2-52ef-4d3e-b106-007dce3ee4ca admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.468712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 927/1889] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:21 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:21.469735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 963/1890] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:21 2022] POST /identity/v3/projects => generated 381 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:21.470483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.472718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:21.477253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972986c2-52ef-4d3e-b106-007dce3ee4ca admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.563147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206353fd-d1a0-4bf5-ac60-c976934b536d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.567380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972986c2-52ef-4d3e-b106-007dce3ee4ca admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.578735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577730d-90a5-4791-b5f1-72065ce2c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.578735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577730d-90a5-4791-b5f1-72065ce2c512 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.578735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577730d-90a5-4791-b5f1-72065ce2c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.578735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577730d-90a5-4791-b5f1-72065ce2c512 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.579076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577730d-90a5-4791-b5f1-72065ce2c512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.579076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577730d-90a5-4791-b5f1-72065ce2c512 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.602667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:21.620412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c2dc6-cc4c-446a-af97-e2466e998473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.621032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2dc6-cc4c-446a-af97-e2466e998473 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.621313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2dc6-cc4c-446a-af97-e2466e998473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.621583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2dc6-cc4c-446a-af97-e2466e998473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.631433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c2dc6-cc4c-446a-af97-e2466e998473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.632988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c2dc6-cc4c-446a-af97-e2466e998473 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.644322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 928/1891] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:21.645208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c2dc6-cc4c-446a-af97-e2466e998473 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.655362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c2dc6-cc4c-446a-af97-e2466e998473 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.661224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9deb6-5759-4a8c-b2a5-2bcbac3c2922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.662095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9deb6-5759-4a8c-b2a5-2bcbac3c2922 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.662672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9deb6-5759-4a8c-b2a5-2bcbac3c2922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.663129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9deb6-5759-4a8c-b2a5-2bcbac3c2922 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.663948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9deb6-5759-4a8c-b2a5-2bcbac3c2922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.671718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9deb6-5759-4a8c-b2a5-2bcbac3c2922 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.688097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 964/1892] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:21 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:21.695548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577730d-90a5-4791-b5f1-72065ce2c512 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.711668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577730d-90a5-4791-b5f1-72065ce2c512 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.717060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bbcf3-34c1-4b1c-823e-b49b9dd413f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.717656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bbcf3-34c1-4b1c-823e-b49b9dd413f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.717939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bbcf3-34c1-4b1c-823e-b49b9dd413f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.722073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bbcf3-34c1-4b1c-823e-b49b9dd413f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.722073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bbcf3-34c1-4b1c-823e-b49b9dd413f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.722073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bbcf3-34c1-4b1c-823e-b49b9dd413f3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.734870 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 965/1893] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:21.736250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bbcf3-34c1-4b1c-823e-b49b9dd413f3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.759875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bbcf3-34c1-4b1c-823e-b49b9dd413f3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.775827 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f73956-c217-4749-9c72-cb122f7db464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.776709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f73956-c217-4749-9c72-cb122f7db464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.777275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f73956-c217-4749-9c72-cb122f7db464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.777660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f73956-c217-4749-9c72-cb122f7db464 admin admin] PATH_INFO: `/v3/projects/829399440f0c4c2ca06a9945db101169/users/d0636008cd3e496a8218f2da780801e9/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.787172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 929/1894] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:21 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:21.791395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9deb6-5759-4a8c-b2a5-2bcbac3c2922 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.812539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f73956-c217-4749-9c72-cb122f7db464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=829399440f0c4c2ca06a9945db101169, user_id=d0636008cd3e496a8218f2da780801e9, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.814875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9deb6-5759-4a8c-b2a5-2bcbac3c2922 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.816530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f73956-c217-4749-9c72-cb122f7db464 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.821588 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681d3f6-5874-49d9-9e7d-4eb7e785ee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.823692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681d3f6-5874-49d9-9e7d-4eb7e785ee4e admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.824140 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681d3f6-5874-49d9-9e7d-4eb7e785ee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.827366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681d3f6-5874-49d9-9e7d-4eb7e785ee4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.827366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681d3f6-5874-49d9-9e7d-4eb7e785ee4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.828337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681d3f6-5874-49d9-9e7d-4eb7e785ee4e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.844145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 930/1895] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:21.847439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681d3f6-5874-49d9-9e7d-4eb7e785ee4e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.859222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681d3f6-5874-49d9-9e7d-4eb7e785ee4e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:21.866952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84491-90fd-42e0-9d97-91acfa8076b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:21.867922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84491-90fd-42e0-9d97-91acfa8076b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:21.868357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84491-90fd-42e0-9d97-91acfa8076b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:21.868820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84491-90fd-42e0-9d97-91acfa8076b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:21.869821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe84491-90fd-42e0-9d97-91acfa8076b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:21.872427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe84491-90fd-42e0-9d97-91acfa8076b6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:21.887695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 931/1896] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:21.893711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84491-90fd-42e0-9d97-91acfa8076b6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.902835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 966/1897] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:21 2022] PUT /identity/v3/projects/829399440f0c4c2ca06a9945db101169/users/d0636008cd3e496a8218f2da780801e9/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:21.907466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f73956-c217-4749-9c72-cb122f7db464 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:21.919168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84491-90fd-42e0-9d97-91acfa8076b6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:22.021401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a6958-9fa3-4006-a170-7748ff7c4ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:22.022079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6958-9fa3-4006-a170-7748ff7c4ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.022436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6958-9fa3-4006-a170-7748ff7c4ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.022637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6958-9fa3-4006-a170-7748ff7c4ae6 admin admin] PATH_INFO: `/v3/projects/034ad8df17234e568a29d3ccb804cab5/users/088d3f1f23c4418ab53027039c367c84/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.043059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a6958-9fa3-4006-a170-7748ff7c4ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=034ad8df17234e568a29d3ccb804cab5, user_id=088d3f1f23c4418ab53027039c367c84, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:22.046298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a6958-9fa3-4006-a170-7748ff7c4ae6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:22.057616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f73956-c217-4749-9c72-cb122f7db464 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:22.063463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064d67c-ccd5-4660-96e3-379b9f599203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:22.064084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064d67c-ccd5-4660-96e3-379b9f599203 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.064373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064d67c-ccd5-4660-96e3-379b9f599203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.064638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064d67c-ccd5-4660-96e3-379b9f599203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.065298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064d67c-ccd5-4660-96e3-379b9f599203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:22.066884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064d67c-ccd5-4660-96e3-379b9f599203 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:22.080933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 967/1898] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:22.085848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064d67c-ccd5-4660-96e3-379b9f599203 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:22.101290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064d67c-ccd5-4660-96e3-379b9f599203 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:22.108067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc40f53-e911-4873-a55e-380aa62c0c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:22.109196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc40f53-e911-4873-a55e-380aa62c0c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.110011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc40f53-e911-4873-a55e-380aa62c0c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.111765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc40f53-e911-4873-a55e-380aa62c0c10 admin admin] PATH_INFO: `/v3/projects/829399440f0c4c2ca06a9945db101169/users/d0636008cd3e496a8218f2da780801e9/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.114245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc40f53-e911-4873-a55e-380aa62c0c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=829399440f0c4c2ca06a9945db101169, user_id=d0636008cd3e496a8218f2da780801e9, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:22.118542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc40f53-e911-4873-a55e-380aa62c0c10 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:22.126440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 932/1899] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:21 2022] PUT /identity/v3/projects/034ad8df17234e568a29d3ccb804cab5/users/088d3f1f23c4418ab53027039c367c84/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:22.134989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 968/1900] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:22 2022] PUT /identity/v3/projects/829399440f0c4c2ca06a9945db101169/users/d0636008cd3e496a8218f2da780801e9/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:22.139654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0195db-dbae-426d-b871-581854a7f221 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.139943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0195db-dbae-426d-b871-581854a7f221 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.140227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0195db-dbae-426d-b871-581854a7f221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.150471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f26da-7aa7-47db-9e4c-ae63cb3bb3bb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.150863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f26da-7aa7-47db-9e4c-ae63cb3bb3bb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.155237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f26da-7aa7-47db-9e4c-ae63cb3bb3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.200605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-718f26da-7aa7-47db-9e4c-ae63cb3bb3bb None None] MFA Rules not processed for user `d0636008cd3e496a8218f2da780801e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:22.201501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0a0195db-dbae-426d-b871-581854a7f221 None None] MFA Rules not processed for user `088d3f1f23c4418ab53027039c367c84`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:22.202729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-718f26da-7aa7-47db-9e4c-ae63cb3bb3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:22.203251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0a0195db-dbae-426d-b871-581854a7f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:22.289122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 969/1901] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:22.297328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 933/1902] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:22.300687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f26da-7aa7-47db-9e4c-ae63cb3bb3bb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:22.372700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f26da-7aa7-47db-9e4c-ae63cb3bb3bb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:22.378733 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:22.379377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.379663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.380051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.380647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:22.382756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:22.447416 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0195db-dbae-426d-b871-581854a7f221 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:22.448983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0a0195db-dbae-426d-b871-581854a7f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:22.476094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0195db-dbae-426d-b871-581854a7f221 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:22.501343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:22.501343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.501343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.501343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] PATH_INFO: `/v3/projects/7dd9359b8d724ec9ac0fdeeb305ee352` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.504255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dd9359b8d724ec9ac0fdeeb305ee352)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:22.506569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:22.586989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 970/1903] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:22 2022] POST /identity/v3/projects => generated 381 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:22.587944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:22.589683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:22.656285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9418a-5ce7-47c0-bd94-2d08ab67c2e6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:22.659920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:22.660523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.660806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.661081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.675979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dd9359b8d724ec9ac0fdeeb305ee352', 'request_id': 'req-816017ab-4b81-45ef-af80-b670fcdfed14'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:22.731354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:22.737089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:22.743419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 971/1904] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:22.744364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:22.745823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:22.786649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dd9359b8d724ec9ac0fdeeb305ee352', 'request_id': 'req-816017ab-4b81-45ef-af80-b670fcdfed14'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:22.789275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:22.795217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dd9359b8d724ec9ac0fdeeb305ee352', 'request_id': 'req-816017ab-4b81-45ef-af80-b670fcdfed14'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:22.821957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] The token cache is being invalidate because project 7dd9359b8d724ec9ac0fdeeb305ee352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:22.822249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:22.829172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:22.830658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 934/1905] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:22 2022] DELETE /identity/v3/projects/7dd9359b8d724ec9ac0fdeeb305ee352 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:22.833915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:22.837179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:22.844248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b332-9dd2-4108-81e0-c72ce43558d1 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:22.918530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816017ab-4b81-45ef-af80-b670fcdfed14 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:22.923869 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:22.924490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.924941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.925228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.925861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:22.926088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:22.926837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:22.927415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:22.927795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] PATH_INFO: `/v3/users/a858cdde46ec4bf8a0ef98f72f363be3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:22.928292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:22.929848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a858cdde46ec4bf8a0ef98f72f363be3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:22.932002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:23.038817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 935/1906] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:22 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:23.039730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:23.046162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:23.064642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca8630-8d52-4adc-8f0f-9fff0c015fb6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:23.073960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:23.074717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:23.075254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:23.075895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] PATH_INFO: `/v3/users/4671ed776399454f811e08832fc712f3` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:23.077327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4671ed776399454f811e08832fc712f3)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:23.079787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:23.213148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a858cdde46ec4bf8a0ef98f72f363be3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.220024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a858cdde46ec4bf8a0ef98f72f363be3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.234862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a858cdde46ec4bf8a0ef98f72f363be3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.235714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a858cdde46ec4bf8a0ef98f72f363be3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.236954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a858cdde46ec4bf8a0ef98f72f363be3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.365971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 972/1907] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:22 2022] DELETE /identity/v3/users/a858cdde46ec4bf8a0ef98f72f363be3 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:23.370722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:23.370722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:23.378777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4671ed776399454f811e08832fc712f3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.396833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4671ed776399454f811e08832fc712f3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.400704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4671ed776399454f811e08832fc712f3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.401562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4671ed776399454f811e08832fc712f3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.402218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4671ed776399454f811e08832fc712f3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:23.444857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea3e1b-dbd8-4996-8a68-0adc3d0b1288 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:23.448854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89de836-92f2-4ab2-8066-f9fcac003021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:23.449464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de836-92f2-4ab2-8066-f9fcac003021 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:23.449766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de836-92f2-4ab2-8066-f9fcac003021 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:23.450036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de836-92f2-4ab2-8066-f9fcac003021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:23.451981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b89de836-92f2-4ab2-8066-f9fcac003021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:23.473433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 936/1908] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:23 2022] DELETE /identity/v3/users/4671ed776399454f811e08832fc712f3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:23.474666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:23.476008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:23.528533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89de836-92f2-4ab2-8066-f9fcac003021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:23.530115 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89de836-92f2-4ab2-8066-f9fcac003021 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:23.542916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 973/1909] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:23.544435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7b9b-e4c6-4598-9f4f-63f1e61f6e91 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:23.544725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7b9b-e4c6-4598-9f4f-63f1e61f6e91 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:23.545003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7b9b-e4c6-4598-9f4f-63f1e61f6e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:23.569117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f6ff7b9b-e4c6-4598-9f4f-63f1e61f6e91 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:23.571180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f6ff7b9b-e4c6-4598-9f4f-63f1e61f6e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:23.636449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976680a-aaa5-4fe3-ab83-84c2086db588 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:23.650360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9cd4-4e91-4c38-b02e-0e1c05edc826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:23.651242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9cd4-4e91-4c38-b02e-0e1c05edc826 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:23.652288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9cd4-4e91-4c38-b02e-0e1c05edc826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:23.652725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9cd4-4e91-4c38-b02e-0e1c05edc826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:23.655319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d9cd4-4e91-4c38-b02e-0e1c05edc826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:23.656972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d9cd4-4e91-4c38-b02e-0e1c05edc826 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:23.677560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 937/1910] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:23.682912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9cd4-4e91-4c38-b02e-0e1c05edc826 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:23.693377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 974/1911] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:23.699547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9cd4-4e91-4c38-b02e-0e1c05edc826 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:23.706829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac4a4-11a2-435e-b30d-4c3d0a0890ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:23.712014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac4a4-11a2-435e-b30d-4c3d0a0890ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:23.712595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac4a4-11a2-435e-b30d-4c3d0a0890ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:23.713014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac4a4-11a2-435e-b30d-4c3d0a0890ee admin admin] PATH_INFO: `/v3/projects/bb9bd2a0b52b402190f9c017c8139531/users/7643d5905e59452aba6b081bfd4463fd/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:23.745436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ac4a4-11a2-435e-b30d-4c3d0a0890ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb9bd2a0b52b402190f9c017c8139531, user_id=7643d5905e59452aba6b081bfd4463fd, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:23.747767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff7b9b-e4c6-4598-9f4f-63f1e61f6e91 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:23.749249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f6ff7b9b-e4c6-4598-9f4f-63f1e61f6e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:23.776839 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ac4a4-11a2-435e-b30d-4c3d0a0890ee admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:23.897538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 938/1912] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:23 2022] PUT /identity/v3/projects/bb9bd2a0b52b402190f9c017c8139531/users/7643d5905e59452aba6b081bfd4463fd/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:23.899454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac4a4-11a2-435e-b30d-4c3d0a0890ee admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:23.901963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff7b9b-e4c6-4598-9f4f-63f1e61f6e91 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:23.989119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc777a6-174e-4088-834a-57fd105b065a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:23.990523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc777a6-174e-4088-834a-57fd105b065a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:23.994916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc777a6-174e-4088-834a-57fd105b065a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:23.994916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc777a6-174e-4088-834a-57fd105b065a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:24.136102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac4a4-11a2-435e-b30d-4c3d0a0890ee admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:24.143813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e918-ee24-404e-bbec-d9557c6e7c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:24.144699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e918-ee24-404e-bbec-d9557c6e7c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:24.145196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e918-ee24-404e-bbec-d9557c6e7c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:24.145640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e918-ee24-404e-bbec-d9557c6e7c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:24.147178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e918-ee24-404e-bbec-d9557c6e7c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:24.149051 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e918-ee24-404e-bbec-d9557c6e7c6c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:24.165847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 939/1913] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:24.171115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e918-ee24-404e-bbec-d9557c6e7c6c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:24.188874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e918-ee24-404e-bbec-d9557c6e7c6c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:24.198064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:24.199214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:24.200367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:24.200739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] PATH_INFO: `/v3/projects/bb9bd2a0b52b402190f9c017c8139531/users/7643d5905e59452aba6b081bfd4463fd/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:24.203457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc777a6-174e-4088-834a-57fd105b065a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:24.204162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb9bd2a0b52b402190f9c017c8139531, user_id=7643d5905e59452aba6b081bfd4463fd, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:24.205930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc777a6-174e-4088-834a-57fd105b065a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:24.207596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:24.220381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 975/1914] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:24.223669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 940/1915] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:24 2022] PUT /identity/v3/projects/bb9bd2a0b52b402190f9c017c8139531/users/7643d5905e59452aba6b081bfd4463fd/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:24.236992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711ec6-1691-41e5-8184-b260eb2450eb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:24.241104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711ec6-1691-41e5-8184-b260eb2450eb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:24.241104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711ec6-1691-41e5-8184-b260eb2450eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:24.285287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8e711ec6-1691-41e5-8184-b260eb2450eb None None] MFA Rules not processed for user `7643d5905e59452aba6b081bfd4463fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:24.292830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8e711ec6-1691-41e5-8184-b260eb2450eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:24.322872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:24.325449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:24.385092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:24.420295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249f5da-2087-41a8-8226-b9ebe0c1a8bd admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:24.427424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:24.428527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:24.428925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:24.429310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] PATH_INFO: `/v3/users/3fd2163da44f44f8931ac7fa11e11313` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:24.430755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd2163da44f44f8931ac7fa11e11313)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:24.431782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 976/1916] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:24.433047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:24.445460 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e711ec6-1691-41e5-8184-b260eb2450eb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:24.450426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8e711ec6-1691-41e5-8184-b260eb2450eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:24.534841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e711ec6-1691-41e5-8184-b260eb2450eb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:24.540929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db07d6-8dcb-460b-8f81-c49095182dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:24.541800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db07d6-8dcb-460b-8f81-c49095182dff service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:24.542202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db07d6-8dcb-460b-8f81-c49095182dff service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:24.542552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db07d6-8dcb-460b-8f81-c49095182dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:24.557665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db07d6-8dcb-460b-8f81-c49095182dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:24.560112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db07d6-8dcb-460b-8f81-c49095182dff service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:24.575738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 977/1917] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:24.665456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd2163da44f44f8931ac7fa11e11313'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:24.701807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd2163da44f44f8931ac7fa11e11313'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:24.710976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd2163da44f44f8931ac7fa11e11313'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:24.711908 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd2163da44f44f8931ac7fa11e11313'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:24.712528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd2163da44f44f8931ac7fa11e11313'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:24.794298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 941/1918] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:24 2022] DELETE /identity/v3/users/3fd2163da44f44f8931ac7fa11e11313 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:24.798951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93c5a9-c2d4-4075-8d44-05a1a7d734dc None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:24.799509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93c5a9-c2d4-4075-8d44-05a1a7d734dc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:24.799509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93c5a9-c2d4-4075-8d44-05a1a7d734dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:24.824232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8b93c5a9-c2d4-4075-8d44-05a1a7d734dc None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:24.825472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8b93c5a9-c2d4-4075-8d44-05a1a7d734dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:24.906013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 978/1919] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:24.913340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:24.914882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:24.986135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c900-b7e2-448c-9032-dfc50d5ee684 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:24.990031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:24.990823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:24.991121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:24.991435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:25.000895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:25.002339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:25.013442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 942/1920] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:25.041261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93c5a9-c2d4-4075-8d44-05a1a7d734dc None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:25.043125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8b93c5a9-c2d4-4075-8d44-05a1a7d734dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:25.064894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93c5a9-c2d4-4075-8d44-05a1a7d734dc None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:25.070811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:25.071502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:25.071779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:25.072051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] PATH_INFO: `/v3/projects/dd487a44183d4e67a4e7d3694f7a5dad` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:25.073253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd487a44183d4e67a4e7d3694f7a5dad)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:25.076154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:25.097335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:25.099524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:25.126027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31837a-a74c-4f00-b1a3-5e99fb7c00ed service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:25.145090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:25.145903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:25.146469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:25.147107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] PATH_INFO: `/v3/projects/683e0273c73e4a758cda5762fb80682d` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:25.148459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683e0273c73e4a758cda5762fb80682d)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:25.150593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:25.314526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd487a44183d4e67a4e7d3694f7a5dad', 'request_id': 'req-8a016cfd-48eb-4ee3-9e97-d2401f0670df'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.355494 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683e0273c73e4a758cda5762fb80682d', 'request_id': 'req-cce09a98-c4fb-4455-937d-958717f513a5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.392840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd487a44183d4e67a4e7d3694f7a5dad', 'request_id': 'req-8a016cfd-48eb-4ee3-9e97-d2401f0670df'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.401524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd487a44183d4e67a4e7d3694f7a5dad', 'request_id': 'req-8a016cfd-48eb-4ee3-9e97-d2401f0670df'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.414262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] The token cache is being invalidate because project dd487a44183d4e67a4e7d3694f7a5dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:25.414612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.415308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.418650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 979/1921] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:25 2022] DELETE /identity/v3/projects/dd487a44183d4e67a4e7d3694f7a5dad => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:25.422212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:25.424168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:25.495448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683e0273c73e4a758cda5762fb80682d', 'request_id': 'req-cce09a98-c4fb-4455-937d-958717f513a5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.511552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683e0273c73e4a758cda5762fb80682d', 'request_id': 'req-cce09a98-c4fb-4455-937d-958717f513a5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.523472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a016cfd-48eb-4ee3-9e97-d2401f0670df admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:25.528850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] The token cache is being invalidate because project 683e0273c73e4a758cda5762fb80682d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:25.528850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.528850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.529438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 943/1922] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:25 2022] DELETE /identity/v3/projects/683e0273c73e4a758cda5762fb80682d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:25.533311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:25.535617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:25.607660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:25.608448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:25.608916 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:25.609286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] PATH_INFO: `/v3/users/bc6a6aec1a554b81927d28bf86df964e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:25.610560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc6a6aec1a554b81927d28bf86df964e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:25.612904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:25.637074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce09a98-c4fb-4455-937d-958717f513a5 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:25.645654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:25.646591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:25.647095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:25.647508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] PATH_INFO: `/v3/users/ed37a87cea5e4714883d8b5515a4523c` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:25.648945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed37a87cea5e4714883d8b5515a4523c)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:25.651117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:25.825859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc6a6aec1a554b81927d28bf86df964e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.843697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc6a6aec1a554b81927d28bf86df964e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.855537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed37a87cea5e4714883d8b5515a4523c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.863237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc6a6aec1a554b81927d28bf86df964e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.865243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc6a6aec1a554b81927d28bf86df964e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.866940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc6a6aec1a554b81927d28bf86df964e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.880056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed37a87cea5e4714883d8b5515a4523c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.898045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed37a87cea5e4714883d8b5515a4523c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.901718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed37a87cea5e4714883d8b5515a4523c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.902722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed37a87cea5e4714883d8b5515a4523c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:25.960327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 980/1923] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:25 2022] DELETE /identity/v3/users/bc6a6aec1a554b81927d28bf86df964e => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:25.969822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:25.972053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:26.062379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 944/1924] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:25 2022] DELETE /identity/v3/users/ed37a87cea5e4714883d8b5515a4523c => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:26.063781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:26.065135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:26.092735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b937-964a-4a21-9c9f-163cfff28373 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:26.192752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9a41-7f2e-41e3-b297-517afa7203f5 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:26.226445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df631a1e-9431-4a97-8b81-c67845a47cfb tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7643d5905e59452aba6b081bfd4463fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb9bd2a0b52b402190f9c017c8139531', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:26.227471 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631a1e-9431-4a97-8b81-c67845a47cfb tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.240322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631a1e-9431-4a97-8b81-c67845a47cfb tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.240322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631a1e-9431-4a97-8b81-c67845a47cfb tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.240641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:26.240641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.240641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.240921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df631a1e-9431-4a97-8b81-c67845a47cfb tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:26.241008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] PATH_INFO: `/v3/projects/ece10fe09dc449529c7e3ccf88f42e70` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.241086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df631a1e-9431-4a97-8b81-c67845a47cfb tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:26.241174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece10fe09dc449529c7e3ccf88f42e70)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:26.241174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:26.241353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df631a1e-9431-4a97-8b81-c67845a47cfb tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:26.262917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 945/1925] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:26.264253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ff28-9bdc-4ca5-985d-5a586c4cdb55 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.264636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ff28-9bdc-4ca5-985d-5a586c4cdb55 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.264801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ff28-9bdc-4ca5-985d-5a586c4cdb55 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.266119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 946/1926] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:22:26.266812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7ff28-9bdc-4ca5-985d-5a586c4cdb55 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:26.281845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7ff28-9bdc-4ca5-985d-5a586c4cdb55 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:26.299025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77c8e6-6b1b-42dd-b8a8-d0bc9d848623 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7643d5905e59452aba6b081bfd4463fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb9bd2a0b52b402190f9c017c8139531', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:26.299025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77c8e6-6b1b-42dd-b8a8-d0bc9d848623 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.299025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77c8e6-6b1b-42dd-b8a8-d0bc9d848623 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.301017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77c8e6-6b1b-42dd-b8a8-d0bc9d848623 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.301017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77c8e6-6b1b-42dd-b8a8-d0bc9d848623 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:26.301017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77c8e6-6b1b-42dd-b8a8-d0bc9d848623 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:26.330916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:26.421412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 947/1927] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:26 2022] POST /identity/v3/projects => generated 354 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:26.422772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bbcde-3e1b-4575-acb9-d3bc6d1ce864 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.423332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bbcde-3e1b-4575-acb9-d3bc6d1ce864 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.424386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bbcde-3e1b-4575-acb9-d3bc6d1ce864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.428978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece10fe09dc449529c7e3ccf88f42e70', 'request_id': 'req-d1f642fa-cd0c-4376-bbab-27dc419b85c2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.472288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-ab9bbcde-3e1b-4575-acb9-d3bc6d1ce864 None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:26.473508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ab9bbcde-3e1b-4575-acb9-d3bc6d1ce864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:26.491423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece10fe09dc449529c7e3ccf88f42e70', 'request_id': 'req-d1f642fa-cd0c-4376-bbab-27dc419b85c2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.500215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece10fe09dc449529c7e3ccf88f42e70', 'request_id': 'req-d1f642fa-cd0c-4376-bbab-27dc419b85c2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.522877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] The token cache is being invalidate because project ece10fe09dc449529c7e3ccf88f42e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:26.523107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.525860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.529493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 981/1928] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:25 2022] DELETE /identity/v3/projects/ece10fe09dc449529c7e3ccf88f42e70 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:26.531586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:26.534591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:26.584813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 948/1929] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:26.586581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bbcde-3e1b-4575-acb9-d3bc6d1ce864 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:26.588272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ab9bbcde-3e1b-4575-acb9-d3bc6d1ce864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:26.675562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f642fa-cd0c-4376-bbab-27dc419b85c2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:26.697025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bbcde-3e1b-4575-acb9-d3bc6d1ce864 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:26.701907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:26.702675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.702960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.703298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] PATH_INFO: `/v3/users/d0636008cd3e496a8218f2da780801e9` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.704429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0636008cd3e496a8218f2da780801e9)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:26.706424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:26.741336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7643d5905e59452aba6b081bfd4463fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb9bd2a0b52b402190f9c017c8139531', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:26.741975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.742257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.742529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.743206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:26.743513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:26.745098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:26.756714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 982/1930] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:26.758738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:26.759224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:26.779221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412a16d-ebe1-4149-b58c-a1523dc0be74 tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:26.799796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:26.799796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.800070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.800344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.814486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:26.816775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:26.831660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 983/1931] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:26.832496 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:26.834539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:26.848510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaa61b-5b6c-4c46-b45d-742b7a5d2ba8 service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:26.853378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:26.854008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:26.854289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:26.854607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] PATH_INFO: `/v3/projects/50a71b32fd0d495ba9e245428f351186` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:26.855770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a71b32fd0d495ba9e245428f351186)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:26.857727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:26.896247 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:26.898239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0636008cd3e496a8218f2da780801e9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.909950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0636008cd3e496a8218f2da780801e9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.917176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0636008cd3e496a8218f2da780801e9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.917887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0636008cd3e496a8218f2da780801e9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.918503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0636008cd3e496a8218f2da780801e9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.966356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a71b32fd0d495ba9e245428f351186', 'request_id': 'req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:26.993373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 949/1932] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:26 2022] DELETE /identity/v3/users/d0636008cd3e496a8218f2da780801e9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:26.996644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:26.996644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:27.056126 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a71b32fd0d495ba9e245428f351186', 'request_id': 'req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.062247 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a71b32fd0d495ba9e245428f351186', 'request_id': 'req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.070438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a13dd-f766-4a6b-94f3-331437c160bc admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:27.076286 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:27.080138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.080138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.080138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] PATH_INFO: `/v3/projects/df2d537ee6044619b90d744ac258b4b1` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.080138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2d537ee6044619b90d744ac258b4b1)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:27.080776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:27.087796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] The token cache is being invalidate because project 50a71b32fd0d495ba9e245428f351186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:27.088036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.088713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.090462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 984/1933] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:26 2022] DELETE /identity/v3/projects/50a71b32fd0d495ba9e245428f351186 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:27.091202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:27.092584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:27.167452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3a8f3-39a7-48da-9161-a9abcf8d21dd admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:27.175814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2d537ee6044619b90d744ac258b4b1', 'request_id': 'req-b92ea441-6d4b-4afd-a786-d45055c1c730'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.240626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7643d5905e59452aba6b081bfd4463fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb9bd2a0b52b402190f9c017c8139531', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:27.241286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.241576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.241936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] PATH_INFO: `/v3/projects/cecf4288c6444eae8347e0c8a1330a55` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.242680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2d537ee6044619b90d744ac258b4b1', 'request_id': 'req-b92ea441-6d4b-4afd-a786-d45055c1c730'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.243175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cecf4288c6444eae8347e0c8a1330a55)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:27.245294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:27.253795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2d537ee6044619b90d744ac258b4b1', 'request_id': 'req-b92ea441-6d4b-4afd-a786-d45055c1c730'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.284492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] The token cache is being invalidate because project df2d537ee6044619b90d744ac258b4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:27.284799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.286150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b92ea441-6d4b-4afd-a786-d45055c1c730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.289228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 950/1934] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:26 2022] DELETE /identity/v3/projects/df2d537ee6044619b90d744ac258b4b1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:27.295371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d1ea8-36cf-406e-8a5e-aa1a24685771 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.295371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d1ea8-36cf-406e-8a5e-aa1a24685771 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.295371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d1ea8-36cf-406e-8a5e-aa1a24685771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.319850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-be8d1ea8-36cf-406e-8a5e-aa1a24685771 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:27.321246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-be8d1ea8-36cf-406e-8a5e-aa1a24685771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:27.405910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 951/1935] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:27.408191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031cd5-a3f7-4795-8828-fda137c67e23 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.408386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cecf4288c6444eae8347e0c8a1330a55', 'request_id': 'req-68021ef3-8c76-4c40-8928-297b4997807e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.409700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031cd5-a3f7-4795-8828-fda137c67e23 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.410253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031cd5-a3f7-4795-8828-fda137c67e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.436933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bc031cd5-a3f7-4795-8828-fda137c67e23 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:27.438428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bc031cd5-a3f7-4795-8828-fda137c67e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:27.501758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cecf4288c6444eae8347e0c8a1330a55', 'request_id': 'req-68021ef3-8c76-4c40-8928-297b4997807e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.510153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cecf4288c6444eae8347e0c8a1330a55', 'request_id': 'req-68021ef3-8c76-4c40-8928-297b4997807e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.534393 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 952/1936] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:27.535684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] The token cache is being invalidate because project cecf4288c6444eae8347e0c8a1330a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:27.535987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.536613 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201439a2-ccaf-4425-b2e6-35e06407ea1b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.536831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.537587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201439a2-ccaf-4425-b2e6-35e06407ea1b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.538089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201439a2-ccaf-4425-b2e6-35e06407ea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.539862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 985/1937] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:27 2022] DELETE /identity/v3/projects/cecf4288c6444eae8347e0c8a1330a55 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:27.540673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:27.548227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:27.567499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68021ef3-8c76-4c40-8928-297b4997807e tempest-VolumeQuotaClassesTest-1342200743 tempest-VolumeQuotaClassesTest-1342200743-project-admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:27.573978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:27.575204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.575204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.575409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] PATH_INFO: `/v3/projects/829399440f0c4c2ca06a9945db101169` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.576480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=829399440f0c4c2ca06a9945db101169)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:27.577695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-201439a2-ccaf-4425-b2e6-35e06407ea1b None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:27.578948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-201439a2-ccaf-4425-b2e6-35e06407ea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:27.579256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:27.617375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 953/1938] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:27.619060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c375a5-28ad-4d0f-aa90-a7683b7222df None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.619419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c375a5-28ad-4d0f-aa90-a7683b7222df None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.619699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c375a5-28ad-4d0f-aa90-a7683b7222df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.643508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c2c375a5-28ad-4d0f-aa90-a7683b7222df None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:27.644740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c2c375a5-28ad-4d0f-aa90-a7683b7222df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:27.655549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 954/1939] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:27.656863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c375a5-28ad-4d0f-aa90-a7683b7222df None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:27.668472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c375a5-28ad-4d0f-aa90-a7683b7222df None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:27.674043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:27.674765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.675157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.675519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.676353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:27.676733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:27.684446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:27.693816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 955/1940] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:27.694722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:27.711399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe5be6-a61c-47e4-8256-fcae7289dc3c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:27.717791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:27.718610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.719157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.719756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.720704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:27.721120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:27.723460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:27.752443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 956/1941] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:27.753338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:27.755492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '829399440f0c4c2ca06a9945db101169', 'request_id': 'req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.766789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cffb00-443b-443d-bf7e-8c4c66f479d1 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:27.838358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '829399440f0c4c2ca06a9945db101169', 'request_id': 'req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.843183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b02ccc-a00a-49e3-b3f3-2c2b27e4c1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:27.843911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b02ccc-a00a-49e3-b3f3-2c2b27e4c1bc admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.851053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b02ccc-a00a-49e3-b3f3-2c2b27e4c1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.851053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b02ccc-a00a-49e3-b3f3-2c2b27e4c1bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.851483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '829399440f0c4c2ca06a9945db101169', 'request_id': 'req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.851592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b02ccc-a00a-49e3-b3f3-2c2b27e4c1bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:27.851592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b02ccc-a00a-49e3-b3f3-2c2b27e4c1bc admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:27.869131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] The token cache is being invalidate because project 829399440f0c4c2ca06a9945db101169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:27.872782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.875036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:27.877814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 986/1942] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:27 2022] DELETE /identity/v3/projects/829399440f0c4c2ca06a9945db101169 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:27.878701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:27.884469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:27.898839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb88efd-e39e-4cb8-98bf-cee6c74514e6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:27.904279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:27.905184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.905682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.906508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:27.907484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:27.909966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:27.973391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 957/1943] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:27 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:27.975132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436cac8-1181-443a-b0a4-aebbd4031315 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:27.975668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436cac8-1181-443a-b0a4-aebbd4031315 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:27.976321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436cac8-1181-443a-b0a4-aebbd4031315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.007538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1436cac8-1181-443a-b0a4-aebbd4031315 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:28.018865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1436cac8-1181-443a-b0a4-aebbd4031315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:28.022344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 987/1944] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:27 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:28.023566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.032880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:28.051784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:28.139638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 958/1945] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:28.142342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436cac8-1181-443a-b0a4-aebbd4031315 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.155101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436cac8-1181-443a-b0a4-aebbd4031315 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.163835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7654f-63c1-4d2f-8a47-34a1159f43d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.164702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7654f-63c1-4d2f-8a47-34a1159f43d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.165193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7654f-63c1-4d2f-8a47-34a1159f43d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.165702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7654f-63c1-4d2f-8a47-34a1159f43d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.166448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7654f-63c1-4d2f-8a47-34a1159f43d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.168467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de191538-a6e7-4355-b9a9-5151e5d8df0f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.168643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7654f-63c1-4d2f-8a47-34a1159f43d1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.182640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af880af-db3f-4819-b5d2-5ffea0429d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.183535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af880af-db3f-4819-b5d2-5ffea0429d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.184037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af880af-db3f-4819-b5d2-5ffea0429d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.184586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af880af-db3f-4819-b5d2-5ffea0429d22 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.187807 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af880af-db3f-4819-b5d2-5ffea0429d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.190092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af880af-db3f-4819-b5d2-5ffea0429d22 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.291208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 959/1946] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:28 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:28.291769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06821830-fee9-43d0-a7cf-f29c15977769 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.292073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06821830-fee9-43d0-a7cf-f29c15977769 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.292624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06821830-fee9-43d0-a7cf-f29c15977769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.315865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 988/1947] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:28 2022] POST /identity/v3/users => generated 287 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:28.321214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af880af-db3f-4819-b5d2-5ffea0429d22 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.332181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af880af-db3f-4819-b5d2-5ffea0429d22 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.338028 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76142-2de4-4e27-af58-5dd936755474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.338710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76142-2de4-4e27-af58-5dd936755474 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.339033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76142-2de4-4e27-af58-5dd936755474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.339313 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76142-2de4-4e27-af58-5dd936755474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.339969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76142-2de4-4e27-af58-5dd936755474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.342604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76142-2de4-4e27-af58-5dd936755474 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.357446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 989/1948] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:28.358650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76142-2de4-4e27-af58-5dd936755474 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.359578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-06821830-fee9-43d0-a7cf-f29c15977769 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:28.361056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-06821830-fee9-43d0-a7cf-f29c15977769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:28.370823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76142-2de4-4e27-af58-5dd936755474 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.376853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7004a-fabe-4be0-8256-a0c7b91956c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.377476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7004a-fabe-4be0-8256-a0c7b91956c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.378459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7004a-fabe-4be0-8256-a0c7b91956c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.378776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7004a-fabe-4be0-8256-a0c7b91956c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.379626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7004a-fabe-4be0-8256-a0c7b91956c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.381994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 960/1949] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:28.381994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06821830-fee9-43d0-a7cf-f29c15977769 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.387559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7004a-fabe-4be0-8256-a0c7b91956c6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.394006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06821830-fee9-43d0-a7cf-f29c15977769 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.398412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 990/1950] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:28.399574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7004a-fabe-4be0-8256-a0c7b91956c6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.403925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b807c-ee6b-4f1f-95a8-c0ba0a683ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.404892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b807c-ee6b-4f1f-95a8-c0ba0a683ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.405373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b807c-ee6b-4f1f-95a8-c0ba0a683ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.405750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b807c-ee6b-4f1f-95a8-c0ba0a683ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.406729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b807c-ee6b-4f1f-95a8-c0ba0a683ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.409354 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b807c-ee6b-4f1f-95a8-c0ba0a683ff3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.413665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7004a-fabe-4be0-8256-a0c7b91956c6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.419114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.419937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.425399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.425399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.425399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:28.425399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.425821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 961/1951] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:28.425902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.425973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b807c-ee6b-4f1f-95a8-c0ba0a683ff3 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.438181 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b807c-ee6b-4f1f-95a8-c0ba0a683ff3 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.443183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 991/1952] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:28.443183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.447817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67acbf1-1b49-4ccb-8fc7-40fd75d9104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.448481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67acbf1-1b49-4ccb-8fc7-40fd75d9104e admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.448931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67acbf1-1b49-4ccb-8fc7-40fd75d9104e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.449212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67acbf1-1b49-4ccb-8fc7-40fd75d9104e admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.449878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67acbf1-1b49-4ccb-8fc7-40fd75d9104e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.451559 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67acbf1-1b49-4ccb-8fc7-40fd75d9104e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.456076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb863f2-7834-45b5-b53c-2bca7a06108a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.462004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.462900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.463325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.463712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] PATH_INFO: `/v3/projects/9db7a9e6154d44cf8c82a778394bc700/users/bed75adca69849028d356043a2542505/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.467152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 962/1953] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:28.467152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67acbf1-1b49-4ccb-8fc7-40fd75d9104e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.479503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67acbf1-1b49-4ccb-8fc7-40fd75d9104e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.492564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aadb0c-7dd8-4c44-8c3a-f02821ef0dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.493218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aadb0c-7dd8-4c44-8c3a-f02821ef0dea admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.493514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aadb0c-7dd8-4c44-8c3a-f02821ef0dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.493775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aadb0c-7dd8-4c44-8c3a-f02821ef0dea admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.494650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aadb0c-7dd8-4c44-8c3a-f02821ef0dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.497373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aadb0c-7dd8-4c44-8c3a-f02821ef0dea admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.504316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db7a9e6154d44cf8c82a778394bc700, user_id=bed75adca69849028d356043a2542505, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.507679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.590014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 963/1954] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:28 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:28.590457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 992/1955] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:28 2022] PUT /identity/v3/projects/9db7a9e6154d44cf8c82a778394bc700/users/bed75adca69849028d356043a2542505/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:28.591831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.593986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:28.600375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aadb0c-7dd8-4c44-8c3a-f02821ef0dea admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.652043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:28.703780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af027b-3618-4999-b172-95aa1f569b78 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.731452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aadb0c-7dd8-4c44-8c3a-f02821ef0dea admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.737988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091865c4-842e-447a-8d28-0522f70e29a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.739435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091865c4-842e-447a-8d28-0522f70e29a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.740084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091865c4-842e-447a-8d28-0522f70e29a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.740594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091865c4-842e-447a-8d28-0522f70e29a2 admin admin] PATH_INFO: `/v3/projects/eec68de1cf014ccbaa5cd82275a5c1b8/users/0410bbda76f34d098ef6e918cdc26d4b/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.759973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091865c4-842e-447a-8d28-0522f70e29a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec68de1cf014ccbaa5cd82275a5c1b8, user_id=0410bbda76f34d098ef6e918cdc26d4b, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.762834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091865c4-842e-447a-8d28-0522f70e29a2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.782621 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.783299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.783576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.783871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] PATH_INFO: `/v3/users/088d3f1f23c4418ab53027039c367c84` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.784981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=088d3f1f23c4418ab53027039c367c84)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.787256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:28.873794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 964/1956] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:28 2022] PUT /identity/v3/projects/eec68de1cf014ccbaa5cd82275a5c1b8/users/0410bbda76f34d098ef6e918cdc26d4b/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:28.893906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091865c4-842e-447a-8d28-0522f70e29a2 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:28.963584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091865c4-842e-447a-8d28-0522f70e29a2 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:28.968239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ed01-5b0d-4c8a-b01f-c7a9c88d71f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:28.968889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ed01-5b0d-4c8a-b01f-c7a9c88d71f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:28.969170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ed01-5b0d-4c8a-b01f-c7a9c88d71f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:28.969445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ed01-5b0d-4c8a-b01f-c7a9c88d71f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:28.970116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787ed01-5b0d-4c8a-b01f-c7a9c88d71f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:28.972080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787ed01-5b0d-4c8a-b01f-c7a9c88d71f9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.002814 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088d3f1f23c4418ab53027039c367c84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:29.012072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088d3f1f23c4418ab53027039c367c84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:29.025455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088d3f1f23c4418ab53027039c367c84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:29.026241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088d3f1f23c4418ab53027039c367c84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:29.026852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29a244e7-d0e3-41b8-a61c-a6d5ac3ae340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088d3f1f23c4418ab53027039c367c84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:29.079431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 965/1957] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:28 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:29.081403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333839d-31bc-4687-b02e-7eefd16ecb6f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.081909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333839d-31bc-4687-b02e-7eefd16ecb6f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.082020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333839d-31bc-4687-b02e-7eefd16ecb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.105036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 993/1958] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:28 2022] DELETE /identity/v3/users/088d3f1f23c4418ab53027039c367c84 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:29.105036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e043b-92a3-4432-ae56-d9772a807bfa None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.105036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e043b-92a3-4432-ae56-d9772a807bfa None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.105036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e043b-92a3-4432-ae56-d9772a807bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.135607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f333839d-31bc-4687-b02e-7eefd16ecb6f None None] MFA Rules not processed for user `bed75adca69849028d356043a2542505`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:29.136958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f333839d-31bc-4687-b02e-7eefd16ecb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:29.148882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-658e043b-92a3-4432-ae56-d9772a807bfa None None] MFA Rules not processed for user `0410bbda76f34d098ef6e918cdc26d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:29.150113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-658e043b-92a3-4432-ae56-d9772a807bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:29.203401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 966/1959] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:29.204256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333839d-31bc-4687-b02e-7eefd16ecb6f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.205542 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f333839d-31bc-4687-b02e-7eefd16ecb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:29.218917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 994/1960] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:29.220374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d487-3205-4585-a0d6-94186af0023e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.220770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d487-3205-4585-a0d6-94186af0023e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.221183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d487-3205-4585-a0d6-94186af0023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.241732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4235d487-3205-4585-a0d6-94186af0023e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:29.242941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4235d487-3205-4585-a0d6-94186af0023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:29.272804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333839d-31bc-4687-b02e-7eefd16ecb6f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.277938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.278056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.278408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.278611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.279298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.280934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.291919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 967/1961] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:29.292738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.294425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:29.307407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2c34e-783e-44b2-864b-559c779b6de1 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.312672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610add5-77a3-4d24-9f4a-1d980199f392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.313449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610add5-77a3-4d24-9f4a-1d980199f392 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.313556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 995/1962] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:29.314299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d487-3205-4585-a0d6-94186af0023e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.315368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610add5-77a3-4d24-9f4a-1d980199f392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.315634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4235d487-3205-4585-a0d6-94186af0023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:29.316444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610add5-77a3-4d24-9f4a-1d980199f392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.317506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610add5-77a3-4d24-9f4a-1d980199f392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.319967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610add5-77a3-4d24-9f4a-1d980199f392 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.328942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d487-3205-4585-a0d6-94186af0023e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.334898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.335706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.335947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.336231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.337006 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.338950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.391281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 968/1963] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:29 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:29.394882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610add5-77a3-4d24-9f4a-1d980199f392 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.430332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 996/1964] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:29 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:29.431268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.437278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:29.475442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:29.539521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610add5-77a3-4d24-9f4a-1d980199f392 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.573173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b18fb-9b1b-4e6d-b3dd-ea0929d2d9fc admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.586387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc0356-39c0-44e2-acb5-3471efd074d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.587129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc0356-39c0-44e2-acb5-3471efd074d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.587487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc0356-39c0-44e2-acb5-3471efd074d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.587831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc0356-39c0-44e2-acb5-3471efd074d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.616625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e62774-3493-4011-a25e-98a63d1d984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.617421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62774-3493-4011-a25e-98a63d1d984d admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.618461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62774-3493-4011-a25e-98a63d1d984d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.618461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62774-3493-4011-a25e-98a63d1d984d admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.619107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e62774-3493-4011-a25e-98a63d1d984d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.620936 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e62774-3493-4011-a25e-98a63d1d984d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.640585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 969/1965] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:29.641679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e62774-3493-4011-a25e-98a63d1d984d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.656504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e62774-3493-4011-a25e-98a63d1d984d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.661880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d9b87-113f-48b5-a8be-406bbaa1d866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.663044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d9b87-113f-48b5-a8be-406bbaa1d866 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.663331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d9b87-113f-48b5-a8be-406bbaa1d866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.663747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d9b87-113f-48b5-a8be-406bbaa1d866 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.664749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d9b87-113f-48b5-a8be-406bbaa1d866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.664866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc0356-39c0-44e2-acb5-3471efd074d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.666131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc0356-39c0-44e2-acb5-3471efd074d4 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.666545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d9b87-113f-48b5-a8be-406bbaa1d866 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.681932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 997/1966] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:29.681932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc0356-39c0-44e2-acb5-3471efd074d4 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.689193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc0356-39c0-44e2-acb5-3471efd074d4 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.695590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2664336-67f6-4353-a0ae-e5588b694849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.696587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664336-67f6-4353-a0ae-e5588b694849 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.697202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664336-67f6-4353-a0ae-e5588b694849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.697635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664336-67f6-4353-a0ae-e5588b694849 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.698415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2664336-67f6-4353-a0ae-e5588b694849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.700826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2664336-67f6-4353-a0ae-e5588b694849 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.797811 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 970/1967] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:29 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:29.797811 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d9b87-113f-48b5-a8be-406bbaa1d866 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.810261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d9b87-113f-48b5-a8be-406bbaa1d866 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.816604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a15f18-df52-4f74-aaa5-4f07d655eb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.817324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a15f18-df52-4f74-aaa5-4f07d655eb2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.817991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a15f18-df52-4f74-aaa5-4f07d655eb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.818195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a15f18-df52-4f74-aaa5-4f07d655eb2f admin admin] PATH_INFO: `/v3/projects/4f6d221b7ae647669233f6845c19eddc/users/ee59f6d4245f493aa321bc5623f0a528/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.827724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 998/1968] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:29 2022] POST /identity/v3/users => generated 294 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:29.829656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2664336-67f6-4353-a0ae-e5588b694849 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.841504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a15f18-df52-4f74-aaa5-4f07d655eb2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f6d221b7ae647669233f6845c19eddc, user_id=ee59f6d4245f493aa321bc5623f0a528, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.845294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a15f18-df52-4f74-aaa5-4f07d655eb2f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.851784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2664336-67f6-4353-a0ae-e5588b694849 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:29.870863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c721ee0-95b7-4583-995a-4a258ac0c463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:29.871839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c721ee0-95b7-4583-995a-4a258ac0c463 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:29.871839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c721ee0-95b7-4583-995a-4a258ac0c463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:29.873839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c721ee0-95b7-4583-995a-4a258ac0c463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:29.873839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c721ee0-95b7-4583-995a-4a258ac0c463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:29.874625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c721ee0-95b7-4583-995a-4a258ac0c463 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:29.894801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 999/1969] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:29.896805 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c721ee0-95b7-4583-995a-4a258ac0c463 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.928696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 971/1970] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:29 2022] PUT /identity/v3/projects/4f6d221b7ae647669233f6845c19eddc/users/ee59f6d4245f493aa321bc5623f0a528/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:29.929625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a15f18-df52-4f74-aaa5-4f07d655eb2f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:29.952455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c721ee0-95b7-4583-995a-4a258ac0c463 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.125360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:30.147745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6bd1-54cb-49a9-9afd-e08b955d2d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.148366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6bd1-54cb-49a9-9afd-e08b955d2d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.148710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6bd1-54cb-49a9-9afd-e08b955d2d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.148926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6bd1-54cb-49a9-9afd-e08b955d2d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.149580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca6bd1-54cb-49a9-9afd-e08b955d2d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.151213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca6bd1-54cb-49a9-9afd-e08b955d2d50 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.151454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a15f18-df52-4f74-aaa5-4f07d655eb2f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.161918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8553532-f6db-4386-8986-883f5b9e26bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.162839 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8553532-f6db-4386-8986-883f5b9e26bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.162951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8553532-f6db-4386-8986-883f5b9e26bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.165334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8553532-f6db-4386-8986-883f5b9e26bf admin admin] PATH_INFO: `/v3/projects/27bf9bf1be5e49fb8893bd86c211b1aa/users/9b8fc5ad45c244998fa0654cc6de7eaa/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.184472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1000/1971] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:30.185788 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9e79e-4616-4342-9350-4684a8fe87d6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.186070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9e79e-4616-4342-9350-4684a8fe87d6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.186355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9e79e-4616-4342-9350-4684a8fe87d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.189275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8553532-f6db-4386-8986-883f5b9e26bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bf9bf1be5e49fb8893bd86c211b1aa, user_id=9b8fc5ad45c244998fa0654cc6de7eaa, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.195045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8553532-f6db-4386-8986-883f5b9e26bf admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.226264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7aa9e79e-4616-4342-9350-4684a8fe87d6 None None] MFA Rules not processed for user `ee59f6d4245f493aa321bc5623f0a528`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:30.227666 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7aa9e79e-4616-4342-9350-4684a8fe87d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:30.263582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 972/1972] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:29 2022] PUT /identity/v3/projects/27bf9bf1be5e49fb8893bd86c211b1aa/users/9b8fc5ad45c244998fa0654cc6de7eaa/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:30.264662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8553532-f6db-4386-8986-883f5b9e26bf admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.295913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1001/1973] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:30.297002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9e79e-4616-4342-9350-4684a8fe87d6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.307742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9e79e-4616-4342-9350-4684a8fe87d6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.331806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8553532-f6db-4386-8986-883f5b9e26bf admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.338937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.339260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.339587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.340008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] PATH_INFO: `/v3/projects/2863c99428e94f59a20438d518aecde6/users/5858bec4699243b796830abc053b5447/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.364809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2863c99428e94f59a20438d518aecde6, user_id=5858bec4699243b796830abc053b5447, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.367771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.384089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.385079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.385428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.385799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.386566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.388368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.404703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1002/1974] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:30.405418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.407361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:30.466854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f706ebf-4edf-44df-a879-1494eebfb705 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.470612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e81dc-343e-4224-a84a-1ef40cd9a924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.471241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e81dc-343e-4224-a84a-1ef40cd9a924 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.471523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e81dc-343e-4224-a84a-1ef40cd9a924 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.471803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e81dc-343e-4224-a84a-1ef40cd9a924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.495960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 973/1975] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:30 2022] PUT /identity/v3/projects/2863c99428e94f59a20438d518aecde6/users/5858bec4699243b796830abc053b5447/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:30.500895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.500895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:30.531940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e81dc-343e-4224-a84a-1ef40cd9a924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.533546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e81dc-343e-4224-a84a-1ef40cd9a924 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.542767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1003/1976] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:30.543647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e81dc-343e-4224-a84a-1ef40cd9a924 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.554552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e81dc-343e-4224-a84a-1ef40cd9a924 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.560841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214295-c7a7-4770-810f-87f922cb618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.560841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214295-c7a7-4770-810f-87f922cb618d admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.560841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214295-c7a7-4770-810f-87f922cb618d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.560841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214295-c7a7-4770-810f-87f922cb618d admin admin] PATH_INFO: `/v3/projects/27bf9bf1be5e49fb8893bd86c211b1aa/users/9b8fc5ad45c244998fa0654cc6de7eaa/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.562888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20214295-c7a7-4770-810f-87f922cb618d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bf9bf1be5e49fb8893bd86c211b1aa, user_id=9b8fc5ad45c244998fa0654cc6de7eaa, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.565062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953692-c34c-487f-938e-bd5d8435d126 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.565365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20214295-c7a7-4770-810f-87f922cb618d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.570592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.571254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.571534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.571806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] PATH_INFO: `/v3/projects/034ad8df17234e568a29d3ccb804cab5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.572917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=034ad8df17234e568a29d3ccb804cab5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.574879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.576568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1004/1977] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:30 2022] PUT /identity/v3/projects/27bf9bf1be5e49fb8893bd86c211b1aa/users/9b8fc5ad45c244998fa0654cc6de7eaa/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:30.577512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214295-c7a7-4770-810f-87f922cb618d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.641891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214295-c7a7-4770-810f-87f922cb618d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.646909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8720274-7d5f-4fc1-a8a5-6596d1ade424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.647570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8720274-7d5f-4fc1-a8a5-6596d1ade424 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.647852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8720274-7d5f-4fc1-a8a5-6596d1ade424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.648182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8720274-7d5f-4fc1-a8a5-6596d1ade424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.648877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8720274-7d5f-4fc1-a8a5-6596d1ade424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.650490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8720274-7d5f-4fc1-a8a5-6596d1ade424 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.660574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1005/1978] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:30.661597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8720274-7d5f-4fc1-a8a5-6596d1ade424 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.671348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8720274-7d5f-4fc1-a8a5-6596d1ade424 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.675996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.676614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.677599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.677926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] PATH_INFO: `/v3/users/ee59f6d4245f493aa321bc5623f0a528` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.679075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee59f6d4245f493aa321bc5623f0a528)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.680939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.700893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '034ad8df17234e568a29d3ccb804cab5', 'request_id': 'req-294d4d40-3912-43d5-bc1e-8b6200e39cd2'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.771444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '034ad8df17234e568a29d3ccb804cab5', 'request_id': 'req-294d4d40-3912-43d5-bc1e-8b6200e39cd2'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.777753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '034ad8df17234e568a29d3ccb804cab5', 'request_id': 'req-294d4d40-3912-43d5-bc1e-8b6200e39cd2'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.790077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] The token cache is being invalidate because project 034ad8df17234e568a29d3ccb804cab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:30.790371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.794362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-294d4d40-3912-43d5-bc1e-8b6200e39cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.794362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 974/1979] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:30 2022] DELETE /identity/v3/projects/034ad8df17234e568a29d3ccb804cab5 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:30.794362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f65934-6b6d-4558-bc6f-ce71acf142ba None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.794362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f65934-6b6d-4558-bc6f-ce71acf142ba None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.794802 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f65934-6b6d-4558-bc6f-ce71acf142ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.824038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-77f65934-6b6d-4558-bc6f-ce71acf142ba None None] MFA Rules not processed for user `9b8fc5ad45c244998fa0654cc6de7eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:30.825257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-77f65934-6b6d-4558-bc6f-ce71acf142ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:30.893452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 975/1980] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:30.894332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f65934-6b6d-4558-bc6f-ce71acf142ba None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.895871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-77f65934-6b6d-4558-bc6f-ce71acf142ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:30.910414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee59f6d4245f493aa321bc5623f0a528'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.917975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee59f6d4245f493aa321bc5623f0a528'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.930521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee59f6d4245f493aa321bc5623f0a528'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.931310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee59f6d4245f493aa321bc5623f0a528'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.931960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee59f6d4245f493aa321bc5623f0a528'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:30.963388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f65934-6b6d-4558-bc6f-ce71acf142ba None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:30.968604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:30.969302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:30.969578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:30.969850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] PATH_INFO: `/v3/projects/2863c99428e94f59a20438d518aecde6/users/5858bec4699243b796830abc053b5447/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:30.971925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2863c99428e94f59a20438d518aecde6, user_id=5858bec4699243b796830abc053b5447, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:30.977714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:30.995092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 976/1981] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:30 2022] PUT /identity/v3/projects/2863c99428e94f59a20438d518aecde6/users/5858bec4699243b796830abc053b5447/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:30.995092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:30.995092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.042558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1006/1982] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:30 2022] DELETE /identity/v3/users/ee59f6d4245f493aa321bc5623f0a528 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:31.043500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:31.044879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.064430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4400e6-e33c-473e-87b0-2e161ad27ae3 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:31.115743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e528b-f1cb-47c1-959e-e7df4c39cf17 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:31.125561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd9ee7-16be-4623-babd-f2b1ef3da596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:31.126168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:31.126292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd9ee7-16be-4623-babd-f2b1ef3da596 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:31.126292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd9ee7-16be-4623-babd-f2b1ef3da596 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:31.126566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:31.126662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd9ee7-16be-4623-babd-f2b1ef3da596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:31.126860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:31.127184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] PATH_INFO: `/v3/users/7643d5905e59452aba6b081bfd4463fd` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:31.128294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7fcd9ee7-16be-4623-babd-f2b1ef3da596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.129939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7643d5905e59452aba6b081bfd4463fd)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:31.132367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:31.211564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd9ee7-16be-4623-babd-f2b1ef3da596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:31.213015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd9ee7-16be-4623-babd-f2b1ef3da596 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:31.223262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1007/1983] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:31.225143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5fb86-6444-4861-ac33-f0010e8d39a2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:31.225523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5fb86-6444-4861-ac33-f0010e8d39a2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:31.225864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5fb86-6444-4861-ac33-f0010e8d39a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:31.264219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-dfe5fb86-6444-4861-ac33-f0010e8d39a2 None None] MFA Rules not processed for user `5858bec4699243b796830abc053b5447`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:31.265580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dfe5fb86-6444-4861-ac33-f0010e8d39a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.348553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1008/1984] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:31.351278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264325-0f94-47cb-89f5-9a16c31899f2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:31.351577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264325-0f94-47cb-89f5-9a16c31899f2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:31.351857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264325-0f94-47cb-89f5-9a16c31899f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:31.373113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3c264325-0f94-47cb-89f5-9a16c31899f2 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:31.374465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3c264325-0f94-47cb-89f5-9a16c31899f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.382630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:31.383369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1009/1985] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:31.398620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c264325-0f94-47cb-89f5-9a16c31899f2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:31.400048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3c264325-0f94-47cb-89f5-9a16c31899f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.426221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7643d5905e59452aba6b081bfd4463fd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:31.438639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7643d5905e59452aba6b081bfd4463fd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:31.445737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7643d5905e59452aba6b081bfd4463fd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:31.446456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7643d5905e59452aba6b081bfd4463fd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:31.447111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7643d5905e59452aba6b081bfd4463fd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:31.465097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c264325-0f94-47cb-89f5-9a16c31899f2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:31.468023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:31.475026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:31.475692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:31.475997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:31.477838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:31.479743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.497256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:31.498692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:31.509468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1010/1986] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:31.510448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:31.516583 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.531665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9157201-67f6-449a-9202-83d99c85cceb service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:31.537485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:31.538096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:31.538380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:31.538655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:31.541373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:31.562250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 977/1987] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:30 2022] DELETE /identity/v3/users/7643d5905e59452aba6b081bfd4463fd => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:31.622213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:31.624298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:31.641335 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1011/1988] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:32.203294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:32.203428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:32.205393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:32.206992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:32.237401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8d94d-0340-448b-9494-175385bde673 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:32.310862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0ab37-5e6c-4f8d-8223-effa966077d0 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:32.316846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:32.323320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:32.324448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.327146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.327146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] PATH_INFO: `/v3/projects/4f6d221b7ae647669233f6845c19eddc` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.327146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f6d221b7ae647669233f6845c19eddc)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:32.329451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:32.332755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.333823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.334293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] PATH_INFO: `/v3/projects/bb9bd2a0b52b402190f9c017c8139531` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.335703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb9bd2a0b52b402190f9c017c8139531)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:32.340340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:32.489952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f6d221b7ae647669233f6845c19eddc', 'request_id': 'req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.506337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb9bd2a0b52b402190f9c017c8139531', 'request_id': 'req-5dc3ff6f-3880-4444-8867-0593f9fd594f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.616732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f6d221b7ae647669233f6845c19eddc', 'request_id': 'req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.625130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f6d221b7ae647669233f6845c19eddc', 'request_id': 'req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.641818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] The token cache is being invalidate because project 4f6d221b7ae647669233f6845c19eddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:32.642114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.642769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.648844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1012/1989] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:32 2022] DELETE /identity/v3/projects/4f6d221b7ae647669233f6845c19eddc => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:32.648844 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:32.653736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:32.702004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb9bd2a0b52b402190f9c017c8139531', 'request_id': 'req-5dc3ff6f-3880-4444-8867-0593f9fd594f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.710150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb9bd2a0b52b402190f9c017c8139531', 'request_id': 'req-5dc3ff6f-3880-4444-8867-0593f9fd594f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.719504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b987ac-acd5-4fe5-bbcc-168dc6b59a43 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:32.728670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] The token cache is being invalidate because project bb9bd2a0b52b402190f9c017c8139531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:32.729357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.730329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5dc3ff6f-3880-4444-8867-0593f9fd594f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:32.732726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 978/1990] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:32 2022] DELETE /identity/v3/projects/bb9bd2a0b52b402190f9c017c8139531 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:32.734941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b01845-d5fd-4dc9-ac80-f62736d3c585 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.735459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b01845-d5fd-4dc9-ac80-f62736d3c585 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.735934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b01845-d5fd-4dc9-ac80-f62736d3c585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.758398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f3b01845-d5fd-4dc9-ac80-f62736d3c585 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:32.759961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f3b01845-d5fd-4dc9-ac80-f62736d3c585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:32.781691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:32.786276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.786276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.786276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] PATH_INFO: `/v3/users/bed75adca69849028d356043a2542505` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.786276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed75adca69849028d356043a2542505)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:32.786276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:32.821598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 979/1991] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:32.823029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941ba59-b8a7-41ad-a369-0e6a6b0db311 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.823399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941ba59-b8a7-41ad-a369-0e6a6b0db311 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.823740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941ba59-b8a7-41ad-a369-0e6a6b0db311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.833950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:32.846608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0941ba59-b8a7-41ad-a369-0e6a6b0db311 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:32.847830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0941ba59-b8a7-41ad-a369-0e6a6b0db311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:32.856329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 980/1992] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:32.857481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42965e5-d1bc-46d9-8e14-e33878d7c963 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.857761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42965e5-d1bc-46d9-8e14-e33878d7c963 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.858032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42965e5-d1bc-46d9-8e14-e33878d7c963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.877508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a42965e5-d1bc-46d9-8e14-e33878d7c963 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:32.878726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a42965e5-d1bc-46d9-8e14-e33878d7c963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:32.887090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 981/1993] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:32.890294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a73b7c-28d7-4755-8376-f3e3da019c72 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.890294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a73b7c-28d7-4755-8376-f3e3da019c72 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.890294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a73b7c-28d7-4755-8376-f3e3da019c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.907647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-51a73b7c-28d7-4755-8376-f3e3da019c72 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:32.908820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-51a73b7c-28d7-4755-8376-f3e3da019c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:32.917579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 982/1994] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:32.918346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a73b7c-28d7-4755-8376-f3e3da019c72 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:32.927620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a73b7c-28d7-4755-8376-f3e3da019c72 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:32.932545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:32.933211 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.933491 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.933761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.934406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:32.934798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:32.936439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:32.950305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 983/1995] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:32.950854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:32.960915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dbc2b-4327-408b-81b3-98a8910ebf5c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:32.965513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:32.965989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:32.966264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:32.966533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:32.967195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:32.967496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:32.969077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:32.982085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 984/1996] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:32.983172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:32.994974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc681fe7-70e0-4eaf-a34f-56cc1b4dd3fa admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:32.999830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c3b80-460b-4db0-985b-62754a4b9dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.000413 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c3b80-460b-4db0-985b-62754a4b9dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.000689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c3b80-460b-4db0-985b-62754a4b9dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.000957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c3b80-460b-4db0-985b-62754a4b9dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.001595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c3b80-460b-4db0-985b-62754a4b9dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.003515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c3b80-460b-4db0-985b-62754a4b9dcb admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.026823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed75adca69849028d356043a2542505'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:33.033796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed75adca69849028d356043a2542505'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:33.047486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed75adca69849028d356043a2542505'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:33.048208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed75adca69849028d356043a2542505'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:33.048967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed75adca69849028d356043a2542505'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:33.107953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1013/1997] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:32 2022] DELETE /identity/v3/users/bed75adca69849028d356043a2542505 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:33.109002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.109909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 985/1998] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:32 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:33.113470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42921855-b040-4a65-a823-bd74e010d72d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.113752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42921855-b040-4a65-a823-bd74e010d72d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.114032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42921855-b040-4a65-a823-bd74e010d72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.116100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:33.149314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-42921855-b040-4a65-a823-bd74e010d72d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:33.150573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-42921855-b040-4a65-a823-bd74e010d72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:33.190332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea263284-9dd2-43d8-a09b-87a81fb9bb4f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.217285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 986/1999] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:33.218320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42921855-b040-4a65-a823-bd74e010d72d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.219609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-42921855-b040-4a65-a823-bd74e010d72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:33.234674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42921855-b040-4a65-a823-bd74e010d72d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.240059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.240695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.240974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.241245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.242004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.243959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.251124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52de18d-18b4-41ec-a245-e630ead18596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.252151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52de18d-18b4-41ec-a245-e630ead18596 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.252376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52de18d-18b4-41ec-a245-e630ead18596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.252769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52de18d-18b4-41ec-a245-e630ead18596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.253542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52de18d-18b4-41ec-a245-e630ead18596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.256271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52de18d-18b4-41ec-a245-e630ead18596 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.370463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 987/2000] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:33 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:33.371341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.372899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:33.405414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1014/2001] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:33 2022] POST /identity/v3/projects => generated 375 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:33.405414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52de18d-18b4-41ec-a245-e630ead18596 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.439392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fc723-da14-47a8-bec8-2f8672c707a3 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.476076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52de18d-18b4-41ec-a245-e630ead18596 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.482987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d733bbd-9c50-43a8-bdb1-97c2706462ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.484322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d733bbd-9c50-43a8-bdb1-97c2706462ff admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.485067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d733bbd-9c50-43a8-bdb1-97c2706462ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.485067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d733bbd-9c50-43a8-bdb1-97c2706462ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.485583 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d733bbd-9c50-43a8-bdb1-97c2706462ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.487673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d733bbd-9c50-43a8-bdb1-97c2706462ff admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.494095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2036bbc-c23b-4667-b451-0fdfd7562c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.494789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2036bbc-c23b-4667-b451-0fdfd7562c71 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.495177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2036bbc-c23b-4667-b451-0fdfd7562c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.495571 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2036bbc-c23b-4667-b451-0fdfd7562c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.500815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1015/2002] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:33.501492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d733bbd-9c50-43a8-bdb1-97c2706462ff admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.507218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2036bbc-c23b-4667-b451-0fdfd7562c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.509261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2036bbc-c23b-4667-b451-0fdfd7562c71 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.514992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d733bbd-9c50-43a8-bdb1-97c2706462ff admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.519901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d170ef-4079-4bbd-9cba-544aff480f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.520519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d170ef-4079-4bbd-9cba-544aff480f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.520862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d170ef-4079-4bbd-9cba-544aff480f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.521144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d170ef-4079-4bbd-9cba-544aff480f51 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.527263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 988/2003] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:33.527420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d170ef-4079-4bbd-9cba-544aff480f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.527512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2036bbc-c23b-4667-b451-0fdfd7562c71 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.527601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d170ef-4079-4bbd-9cba-544aff480f51 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.533336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2036bbc-c23b-4667-b451-0fdfd7562c71 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.538864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b131964-28e0-445e-a20e-3025386b4a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.539499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b131964-28e0-445e-a20e-3025386b4a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.539772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b131964-28e0-445e-a20e-3025386b4a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.540045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b131964-28e0-445e-a20e-3025386b4a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.540689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b131964-28e0-445e-a20e-3025386b4a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.542945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b131964-28e0-445e-a20e-3025386b4a2f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.557801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 989/2004] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:33.563184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b131964-28e0-445e-a20e-3025386b4a2f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.577040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b131964-28e0-445e-a20e-3025386b4a2f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.579845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:33.588070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7d9ba-fd35-44d4-8396-5a1442b78965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.588070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7d9ba-fd35-44d4-8396-5a1442b78965 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.588428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7d9ba-fd35-44d4-8396-5a1442b78965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.588832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7d9ba-fd35-44d4-8396-5a1442b78965 admin admin] PATH_INFO: `/v3/projects/9b455c49b9954395b62ebe933b95e2b8/users/1b6bf537e5d3429b98af41857858af56/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.612545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1016/2005] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:33 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:33.617045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d170ef-4079-4bbd-9cba-544aff480f51 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.627500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7d9ba-fd35-44d4-8396-5a1442b78965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b455c49b9954395b62ebe933b95e2b8, user_id=1b6bf537e5d3429b98af41857858af56, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.629839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d170ef-4079-4bbd-9cba-544aff480f51 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.633080 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7d9ba-fd35-44d4-8396-5a1442b78965 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.649237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266626-9a1a-4107-8d9c-f97f5fa410a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.650248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266626-9a1a-4107-8d9c-f97f5fa410a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.650553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266626-9a1a-4107-8d9c-f97f5fa410a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.651166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266626-9a1a-4107-8d9c-f97f5fa410a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.653847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2266626-9a1a-4107-8d9c-f97f5fa410a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.655774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2266626-9a1a-4107-8d9c-f97f5fa410a7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.681721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1017/2006] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:33.682802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266626-9a1a-4107-8d9c-f97f5fa410a7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.694751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266626-9a1a-4107-8d9c-f97f5fa410a7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.700318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b37729-b606-427c-90fb-6b169f954127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.700990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b37729-b606-427c-90fb-6b169f954127 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.701339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b37729-b606-427c-90fb-6b169f954127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.701820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b37729-b606-427c-90fb-6b169f954127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.702533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b37729-b606-427c-90fb-6b169f954127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.704209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b37729-b606-427c-90fb-6b169f954127 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.720381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 990/2007] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:33 2022] PUT /identity/v3/projects/9b455c49b9954395b62ebe933b95e2b8/users/1b6bf537e5d3429b98af41857858af56/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:33.731159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1018/2008] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:33.732191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7d9ba-fd35-44d4-8396-5a1442b78965 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.732778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c452ad-cdb4-4bfb-b7d4-f9ce745f178e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.733231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c452ad-cdb4-4bfb-b7d4-f9ce745f178e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.733573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c452ad-cdb4-4bfb-b7d4-f9ce745f178e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.766388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-23c452ad-cdb4-4bfb-b7d4-f9ce745f178e None None] MFA Rules not processed for user `1b6bf537e5d3429b98af41857858af56`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:33.768081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-23c452ad-cdb4-4bfb-b7d4-f9ce745f178e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:33.808036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7d9ba-fd35-44d4-8396-5a1442b78965 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.821766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f286514-74f7-4f6c-913d-4d76022330bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.823299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f286514-74f7-4f6c-913d-4d76022330bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.823776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f286514-74f7-4f6c-913d-4d76022330bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.824387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f286514-74f7-4f6c-913d-4d76022330bf admin admin] PATH_INFO: `/v3/projects/549f902c93c0487e957156b6744b5f1a/users/444a1ac1fc82456c98e18ae6eb961ce8/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.850619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1019/2009] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:33.871055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f286514-74f7-4f6c-913d-4d76022330bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=549f902c93c0487e957156b6744b5f1a, user_id=444a1ac1fc82456c98e18ae6eb961ce8, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.874036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f286514-74f7-4f6c-913d-4d76022330bf admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.874804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c452ad-cdb4-4bfb-b7d4-f9ce745f178e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:33.876544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-23c452ad-cdb4-4bfb-b7d4-f9ce745f178e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:33.940556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c452ad-cdb4-4bfb-b7d4-f9ce745f178e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:33.946389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a8c94-ca46-4b44-919f-45a9e6684eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:33.946389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8c94-ca46-4b44-919f-45a9e6684eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.946389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8c94-ca46-4b44-919f-45a9e6684eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.946389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8c94-ca46-4b44-919f-45a9e6684eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.955725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a8c94-ca46-4b44-919f-45a9e6684eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:33.957483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a8c94-ca46-4b44-919f-45a9e6684eb9 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:33.970762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 991/2010] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:33 2022] PUT /identity/v3/projects/549f902c93c0487e957156b6744b5f1a/users/444a1ac1fc82456c98e18ae6eb961ce8/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:33.972556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1020/2011] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:33.987666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efae924-3f18-423e-8682-8f694185e513 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:33.987666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efae924-3f18-423e-8682-8f694185e513 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:33.987666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efae924-3f18-423e-8682-8f694185e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:33.998483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a8c94-ca46-4b44-919f-45a9e6684eb9 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:34.014369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a8c94-ca46-4b44-919f-45a9e6684eb9 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:34.037248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8efae924-3f18-423e-8682-8f694185e513 None None] MFA Rules not processed for user `444a1ac1fc82456c98e18ae6eb961ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:34.038770 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8efae924-3f18-423e-8682-8f694185e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:34.091896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:34.092641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:34.092992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:34.093380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] PATH_INFO: `/v3/users/1b6bf537e5d3429b98af41857858af56` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:34.094639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b6bf537e5d3429b98af41857858af56)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:34.096885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:34.130783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 992/2012] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:34.130783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efae924-3f18-423e-8682-8f694185e513 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:34.130783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8efae924-3f18-423e-8682-8f694185e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:34.147855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efae924-3f18-423e-8682-8f694185e513 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:34.153053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:34.160208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:34.160208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:34.160208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] PATH_INFO: `/v3/projects/9db7a9e6154d44cf8c82a778394bc700` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:34.160208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db7a9e6154d44cf8c82a778394bc700)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:34.160208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:34.329616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:34.353293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db7a9e6154d44cf8c82a778394bc700', 'request_id': 'req-94f6f233-bcd4-40a6-b312-284ebc478784'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.354810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b6bf537e5d3429b98af41857858af56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.361994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b6bf537e5d3429b98af41857858af56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.376348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b6bf537e5d3429b98af41857858af56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.377151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b6bf537e5d3429b98af41857858af56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.377842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b6bf537e5d3429b98af41857858af56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.478177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db7a9e6154d44cf8c82a778394bc700', 'request_id': 'req-94f6f233-bcd4-40a6-b312-284ebc478784'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.485081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db7a9e6154d44cf8c82a778394bc700', 'request_id': 'req-94f6f233-bcd4-40a6-b312-284ebc478784'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.496736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] The token cache is being invalidate because project 9db7a9e6154d44cf8c82a778394bc700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:34.496962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.497614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.499133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 993/2013] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:34 2022] DELETE /identity/v3/projects/9db7a9e6154d44cf8c82a778394bc700 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:34.500000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:34.501266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:34.563899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f233-bcd4-40a6-b312-284ebc478784 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:34.568910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7be9e-ed7f-4a15-aace-861796970050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:34.569519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7be9e-ed7f-4a15-aace-861796970050 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:34.569796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7be9e-ed7f-4a15-aace-861796970050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:34.570067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7be9e-ed7f-4a15-aace-861796970050 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:34.570763 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7be9e-ed7f-4a15-aace-861796970050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:34.572786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7be9e-ed7f-4a15-aace-861796970050 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:34.585422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1021/2014] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:33 2022] DELETE /identity/v3/users/1b6bf537e5d3429b98af41857858af56 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:34.589566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:34.591109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:34.656372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b966-5011-43b5-9550-e62071568d20 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:34.661008 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:34.661664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:34.661943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:34.662214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] PATH_INFO: `/v3/users/9b8fc5ad45c244998fa0654cc6de7eaa` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:34.663362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8fc5ad45c244998fa0654cc6de7eaa)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:34.665248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:34.678212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 994/2015] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:34 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:34.679957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccbe01-1429-40b7-881a-527ed2f56b92 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:34.680247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccbe01-1429-40b7-881a-527ed2f56b92 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:34.680520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccbe01-1429-40b7-881a-527ed2f56b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:34.706723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d3ccbe01-1429-40b7-881a-527ed2f56b92 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:34.707567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d3ccbe01-1429-40b7-881a-527ed2f56b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:34.781958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 995/2016] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:34.782793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccbe01-1429-40b7-881a-527ed2f56b92 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:34.797438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccbe01-1429-40b7-881a-527ed2f56b92 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:34.800029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:34.800646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:34.800942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:34.801219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:34.802456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:34.804794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:34.879457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 996/2017] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:34 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:34.880616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:34.888039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:34.910331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8fc5ad45c244998fa0654cc6de7eaa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.918221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8fc5ad45c244998fa0654cc6de7eaa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.932278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8fc5ad45c244998fa0654cc6de7eaa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.933090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8fc5ad45c244998fa0654cc6de7eaa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.934300 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8fc5ad45c244998fa0654cc6de7eaa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:34.952579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e2b12-e9f9-4e0d-9147-267fb70dc59b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:34.956507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:34.957120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:34.957397 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:34.957715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:34.959585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:34.971551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:34.972982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:34.984144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 997/2018] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:34.985334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:34.988438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:35.008224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d88f-d6b9-432c-957a-8f662dd61d34 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:35.018927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1022/2019] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:34 2022] DELETE /identity/v3/users/9b8fc5ad45c244998fa0654cc6de7eaa => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:35.021638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feab1a-783e-4274-897f-46db5dea5fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:35.022367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feab1a-783e-4274-897f-46db5dea5fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:35.022704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feab1a-783e-4274-897f-46db5dea5fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:35.023060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feab1a-783e-4274-897f-46db5dea5fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:35.026353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52feab1a-783e-4274-897f-46db5dea5fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:35.026353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52feab1a-783e-4274-897f-46db5dea5fd6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:35.030650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:35.032591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:35.046837 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 998/2020] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:35.050088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feab1a-783e-4274-897f-46db5dea5fd6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:35.063597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feab1a-783e-4274-897f-46db5dea5fd6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:35.146156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46932d-5d39-45b9-8058-57ed1a36b319 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:35.151706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:35.152517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:35.152801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:35.153072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] PATH_INFO: `/v3/users/0410bbda76f34d098ef6e918cdc26d4b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:35.154282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0410bbda76f34d098ef6e918cdc26d4b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:35.156227 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:35.184096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97afcb-9887-41f6-9b89-63ce45c08b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:35.185959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97afcb-9887-41f6-9b89-63ce45c08b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:35.186886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97afcb-9887-41f6-9b89-63ce45c08b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:35.186886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97afcb-9887-41f6-9b89-63ce45c08b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:35.191663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97afcb-9887-41f6-9b89-63ce45c08b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:35.194122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97afcb-9887-41f6-9b89-63ce45c08b83 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:35.205145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 999/2021] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:35.209937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97afcb-9887-41f6-9b89-63ce45c08b83 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:35.222203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97afcb-9887-41f6-9b89-63ce45c08b83 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:35.234135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cedfe-7452-432d-8e96-917a4aa61223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:35.234135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cedfe-7452-432d-8e96-917a4aa61223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:35.234135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cedfe-7452-432d-8e96-917a4aa61223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:35.234135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cedfe-7452-432d-8e96-917a4aa61223 admin admin] PATH_INFO: `/v3/projects/1c1a79ce70b0486aa19a7aca35b9c835/users/208372b2aeeb4deea2fa6b74a752b2b5/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:35.245403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36cedfe-7452-432d-8e96-917a4aa61223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c1a79ce70b0486aa19a7aca35b9c835, user_id=208372b2aeeb4deea2fa6b74a752b2b5, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:35.248147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36cedfe-7452-432d-8e96-917a4aa61223 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:35.390087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1000/2022] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:35 2022] PUT /identity/v3/projects/1c1a79ce70b0486aa19a7aca35b9c835/users/208372b2aeeb4deea2fa6b74a752b2b5/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:35.400065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487077e5-1390-4454-8153-22160e083d25 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:35.400522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487077e5-1390-4454-8153-22160e083d25 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:35.400800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487077e5-1390-4454-8153-22160e083d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:35.410304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0410bbda76f34d098ef6e918cdc26d4b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.422816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0410bbda76f34d098ef6e918cdc26d4b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.449552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0410bbda76f34d098ef6e918cdc26d4b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.450291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0410bbda76f34d098ef6e918cdc26d4b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.450889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0410bbda76f34d098ef6e918cdc26d4b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.459084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-487077e5-1390-4454-8153-22160e083d25 None None] MFA Rules not processed for user `208372b2aeeb4deea2fa6b74a752b2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:35.460422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-487077e5-1390-4454-8153-22160e083d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:35.533203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1001/2023] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:35.538435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487077e5-1390-4454-8153-22160e083d25 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:35.539881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-487077e5-1390-4454-8153-22160e083d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:35.605672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487077e5-1390-4454-8153-22160e083d25 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:35.610925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1023/2024] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:35 2022] DELETE /identity/v3/users/0410bbda76f34d098ef6e918cdc26d4b => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:35.610925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:35.611709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:35.687610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b71e7-b97d-45ea-a6cd-6d9e8136f71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:35.695667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b71e7-b97d-45ea-a6cd-6d9e8136f71e admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:35.695979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b71e7-b97d-45ea-a6cd-6d9e8136f71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:35.696271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b71e7-b97d-45ea-a6cd-6d9e8136f71e admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:35.697083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b71e7-b97d-45ea-a6cd-6d9e8136f71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:35.699025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b71e7-b97d-45ea-a6cd-6d9e8136f71e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:35.702800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9ff81-a41e-4e2d-b0ff-721a77d001e9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:35.710418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:35.710418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:35.710418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:35.710418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] PATH_INFO: `/v3/projects/9b455c49b9954395b62ebe933b95e2b8` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:35.712712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:35.714720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b455c49b9954395b62ebe933b95e2b8)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:35.716968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:35.845521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1002/2025] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:35 2022] POST /identity/v3/projects => generated 375 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:35.845521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fcd21-2cd4-4fe6-afd9-5ea66a202d50 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:35.845670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fcd21-2cd4-4fe6-afd9-5ea66a202d50 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:35.845926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fcd21-2cd4-4fe6-afd9-5ea66a202d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:35.857995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b455c49b9954395b62ebe933b95e2b8', 'request_id': 'req-1df33656-259d-4355-be5f-fab2e833a0d7'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.887135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d28fcd21-2cd4-4fe6-afd9-5ea66a202d50 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:35.889438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d28fcd21-2cd4-4fe6-afd9-5ea66a202d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:35.943029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b455c49b9954395b62ebe933b95e2b8', 'request_id': 'req-1df33656-259d-4355-be5f-fab2e833a0d7'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.951012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b455c49b9954395b62ebe933b95e2b8', 'request_id': 'req-1df33656-259d-4355-be5f-fab2e833a0d7'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.968255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] The token cache is being invalidate because project 9b455c49b9954395b62ebe933b95e2b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:35.968887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.969734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:35.969870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1003/2026] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:35.970855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fcd21-2cd4-4fe6-afd9-5ea66a202d50 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:35.974869 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d28fcd21-2cd4-4fe6-afd9-5ea66a202d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:35.982778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1024/2027] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:35 2022] DELETE /identity/v3/projects/9b455c49b9954395b62ebe933b95e2b8 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:35.983925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:35.986021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:36.004553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fcd21-2cd4-4fe6-afd9-5ea66a202d50 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.018484 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.019258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.019439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.019774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] PATH_INFO: `/v3/projects/27bf9bf1be5e49fb8893bd86c211b1aa` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.023544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bf9bf1be5e49fb8893bd86c211b1aa)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.028454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.092175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df33656-259d-4355-be5f-fab2e833a0d7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.097438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda62b40-ef26-4263-a6f9-ebc2413c213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.098784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda62b40-ef26-4263-a6f9-ebc2413c213b admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.098784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda62b40-ef26-4263-a6f9-ebc2413c213b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.099361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda62b40-ef26-4263-a6f9-ebc2413c213b admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.100241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda62b40-ef26-4263-a6f9-ebc2413c213b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.102146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda62b40-ef26-4263-a6f9-ebc2413c213b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.159688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bf9bf1be5e49fb8893bd86c211b1aa', 'request_id': 'req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:36.200120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1025/2028] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:35 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:36.211809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba87c31-c892-42ed-b3e3-0f0b63626f15 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.212174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba87c31-c892-42ed-b3e3-0f0b63626f15 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.212517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba87c31-c892-42ed-b3e3-0f0b63626f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.233179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cba87c31-c892-42ed-b3e3-0f0b63626f15 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:36.234405 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cba87c31-c892-42ed-b3e3-0f0b63626f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:36.273371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bf9bf1be5e49fb8893bd86c211b1aa', 'request_id': 'req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:36.277722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:36.281189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bf9bf1be5e49fb8893bd86c211b1aa', 'request_id': 'req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:36.293665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] The token cache is being invalidate because project 27bf9bf1be5e49fb8893bd86c211b1aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:36.293962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:36.294616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:36.296412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1004/2029] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:35 2022] DELETE /identity/v3/projects/27bf9bf1be5e49fb8893bd86c211b1aa => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:36.296971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.298270 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:36.303175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1026/2030] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:36.304112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba87c31-c892-42ed-b3e3-0f0b63626f15 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.308180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cba87c31-c892-42ed-b3e3-0f0b63626f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:36.333841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba87c31-c892-42ed-b3e3-0f0b63626f15 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.344564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa05e3-c0be-4922-89de-c192ee3fd50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.344941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa05e3-c0be-4922-89de-c192ee3fd50e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.345527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa05e3-c0be-4922-89de-c192ee3fd50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.346019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa05e3-c0be-4922-89de-c192ee3fd50e admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.347572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa05e3-c0be-4922-89de-c192ee3fd50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.349261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa05e3-c0be-4922-89de-c192ee3fd50e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.366591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1027/2031] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:36.368450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c5ee2-fac4-4e75-b9d2-eb5da01cd227 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.368785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c5ee2-fac4-4e75-b9d2-eb5da01cd227 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.369120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c5ee2-fac4-4e75-b9d2-eb5da01cd227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.377078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc60964-2387-4b6f-9f0d-5e84aaa57f6e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.383200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.383906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.384344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.384628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.386766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:36.392619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4c2c5ee2-fac4-4e75-b9d2-eb5da01cd227 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:36.394267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4c2c5ee2-fac4-4e75-b9d2-eb5da01cd227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:36.402139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.404174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.411826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1028/2032] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:36.411826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc780-60f3-43e9-9d72-4480584f64ae None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.411826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc780-60f3-43e9-9d72-4480584f64ae None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.411826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc780-60f3-43e9-9d72-4480584f64ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.419708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1005/2033] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:36.420764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.431598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca04a62-c213-4253-a0a0-6a22c7c64c09 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.431788 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-09bcc780-60f3-43e9-9d72-4480584f64ae None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:36.433079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-09bcc780-60f3-43e9-9d72-4480584f64ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:36.437355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696f050-b8ef-4186-8d05-6aa3fd9ab786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.438033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696f050-b8ef-4186-8d05-6aa3fd9ab786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.438338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696f050-b8ef-4186-8d05-6aa3fd9ab786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.438728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696f050-b8ef-4186-8d05-6aa3fd9ab786 admin admin] PATH_INFO: `/v3/projects/dce1a4765a9941889af243b4f61675e7/users/0def5ff84c724cf4be406945b747990e/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.452467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1029/2034] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:36.456387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7cc94-0728-4b55-a91d-4362f19d5070 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.456676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7cc94-0728-4b55-a91d-4362f19d5070 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.456790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696f050-b8ef-4186-8d05-6aa3fd9ab786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce1a4765a9941889af243b4f61675e7, user_id=0def5ff84c724cf4be406945b747990e, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.458799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7cc94-0728-4b55-a91d-4362f19d5070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.460440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696f050-b8ef-4186-8d05-6aa3fd9ab786 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.486500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cec7cc94-0728-4b55-a91d-4362f19d5070 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:36.487954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cec7cc94-0728-4b55-a91d-4362f19d5070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:36.500865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1030/2035] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:36.501836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7cc94-0728-4b55-a91d-4362f19d5070 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.517557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7cc94-0728-4b55-a91d-4362f19d5070 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.525114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.525742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.526027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.526360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.527056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:36.527393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.529036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.547896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1031/2036] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:36.547896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.556328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2360325-5c33-4f3f-9b12-8fcd442d8eda admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.561062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.561652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.561935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.562207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.562850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:36.563192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.564872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.576792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1006/2037] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:36 2022] PUT /identity/v3/projects/dce1a4765a9941889af243b4f61675e7/users/0def5ff84c724cf4be406945b747990e/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:36.577874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696f050-b8ef-4186-8d05-6aa3fd9ab786 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.578131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1032/2038] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:36.580630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.592293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3710e-e5ce-41c5-aed9-f4f5bbd36c01 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.673023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcef35-a757-4133-b05f-8717f402134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.673646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcef35-a757-4133-b05f-8717f402134f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.673914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcef35-a757-4133-b05f-8717f402134f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.674193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcef35-a757-4133-b05f-8717f402134f admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.675039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcef35-a757-4133-b05f-8717f402134f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.677454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcef35-a757-4133-b05f-8717f402134f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.689345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1033/2039] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:36.689753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696f050-b8ef-4186-8d05-6aa3fd9ab786 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.690229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcef35-a757-4133-b05f-8717f402134f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.697062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684b007-bbc4-4577-972c-feeb11a19799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.697725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684b007-bbc4-4577-972c-feeb11a19799 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.698008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684b007-bbc4-4577-972c-feeb11a19799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.698281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684b007-bbc4-4577-972c-feeb11a19799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.698912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684b007-bbc4-4577-972c-feeb11a19799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.700911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684b007-bbc4-4577-972c-feeb11a19799 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.702334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcef35-a757-4133-b05f-8717f402134f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.708688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4159a-dd9b-4afc-86f0-6c00dc35b091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.709701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4159a-dd9b-4afc-86f0-6c00dc35b091 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.710170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4159a-dd9b-4afc-86f0-6c00dc35b091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.710601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4159a-dd9b-4afc-86f0-6c00dc35b091 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.711415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4159a-dd9b-4afc-86f0-6c00dc35b091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.713856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4159a-dd9b-4afc-86f0-6c00dc35b091 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.812514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:36.821928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1007/2040] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:36 2022] POST /identity/v3/projects => generated 371 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:36.821928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684b007-bbc4-4577-972c-feeb11a19799 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.824151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1034/2041] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:36 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:36.825097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4159a-dd9b-4afc-86f0-6c00dc35b091 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:36.898371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684b007-bbc4-4577-972c-feeb11a19799 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.923514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4159a-dd9b-4afc-86f0-6c00dc35b091 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:36.928578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.929224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.929583 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.929781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.930412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.932400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.965142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049feea8-221d-42d2-92af-894c46be6e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:36.965829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049feea8-221d-42d2-92af-894c46be6e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:36.966113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049feea8-221d-42d2-92af-894c46be6e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:36.966390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049feea8-221d-42d2-92af-894c46be6e9b admin admin] PATH_INFO: `/v3/projects/dce1a4765a9941889af243b4f61675e7/users/0def5ff84c724cf4be406945b747990e/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:36.968424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049feea8-221d-42d2-92af-894c46be6e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce1a4765a9941889af243b4f61675e7, user_id=0def5ff84c724cf4be406945b747990e, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:36.971200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049feea8-221d-42d2-92af-894c46be6e9b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:36.992024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1008/2042] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:36 2022] PUT /identity/v3/projects/dce1a4765a9941889af243b4f61675e7/users/0def5ff84c724cf4be406945b747990e/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:37.001690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049feea8-221d-42d2-92af-894c46be6e9b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.018197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1035/2043] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:36 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:37.018197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.019419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.031846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d9112-54fb-46e8-86b0-37969f68453a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.064989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049feea8-221d-42d2-92af-894c46be6e9b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.070391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d57c2-ca46-4d4e-bb76-ddccbb94a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.071020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d57c2-ca46-4d4e-bb76-ddccbb94a3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.071338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d57c2-ca46-4d4e-bb76-ddccbb94a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.071635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d57c2-ca46-4d4e-bb76-ddccbb94a3b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.072271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d57c2-ca46-4d4e-bb76-ddccbb94a3b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.074202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d57c2-ca46-4d4e-bb76-ddccbb94a3b4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.091038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.091723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.092007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.096729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] PATH_INFO: `/v3/projects/eec68de1cf014ccbaa5cd82275a5c1b8` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.096729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec68de1cf014ccbaa5cd82275a5c1b8)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.096729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.169306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1009/2044] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:36 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:37.170819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34630b3-12cc-4304-acb2-84a833a6aa3a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.171278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34630b3-12cc-4304-acb2-84a833a6aa3a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.172785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34630b3-12cc-4304-acb2-84a833a6aa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.206229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f34630b3-12cc-4304-acb2-84a833a6aa3a None None] MFA Rules not processed for user `0def5ff84c724cf4be406945b747990e`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:37.207541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f34630b3-12cc-4304-acb2-84a833a6aa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.253711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:37.263252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec68de1cf014ccbaa5cd82275a5c1b8', 'request_id': 'req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.302652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1010/2045] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:37.303623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34630b3-12cc-4304-acb2-84a833a6aa3a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.314305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34630b3-12cc-4304-acb2-84a833a6aa3a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.353704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec68de1cf014ccbaa5cd82275a5c1b8', 'request_id': 'req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.362960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec68de1cf014ccbaa5cd82275a5c1b8', 'request_id': 'req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.376624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.380452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.380452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.380452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.380452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.380452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.387025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] The token cache is being invalidate because project eec68de1cf014ccbaa5cd82275a5c1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:37.387025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.387025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.388880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1036/2046] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:37 2022] DELETE /identity/v3/projects/eec68de1cf014ccbaa5cd82275a5c1b8 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:37.391806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.392144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.395041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1011/2047] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:37.395041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.399337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.415990 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0034c-c26c-42f0-916a-1ea2a05ae9f6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.423604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.424209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.424485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.424754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.425403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.427270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.439177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1037/2048] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:37.439177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.444441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.458794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dc764-6a13-4a64-848d-6bb34b6dec64 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.463832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.464622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.464898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.465169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] PATH_INFO: `/v3/users/5858bec4699243b796830abc053b5447` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.466574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5858bec4699243b796830abc053b5447)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.468514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.472281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da8171-0466-4a00-9cf6-19a82e0245cd admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.479279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.480013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.480351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.480746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.483581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.554155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.555645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.565218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1012/2049] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:37.566048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.567447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.582378 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a92433-3adf-486e-96a4-a2370ad2c072 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.587966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91368cf2-7328-4663-a52c-f81bbe47d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.588647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91368cf2-7328-4663-a52c-f81bbe47d988 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.589050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91368cf2-7328-4663-a52c-f81bbe47d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.589332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91368cf2-7328-4663-a52c-f81bbe47d988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.590186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91368cf2-7328-4663-a52c-f81bbe47d988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.592001 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91368cf2-7328-4663-a52c-f81bbe47d988 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.603562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1013/2050] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:37.604666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91368cf2-7328-4663-a52c-f81bbe47d988 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.618724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91368cf2-7328-4663-a52c-f81bbe47d988 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.625492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.626357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.626812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.627278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.628068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.630502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.646002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1014/2051] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:37.646002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.646002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.659604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf75a25-b19e-4356-83f6-bb749c0f4848 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.664468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.665100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.665378 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.665649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] PATH_INFO: `/v3/users/ce72a82d8ed34f4a962be77d4a5c8455` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.666776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce72a82d8ed34f4a962be77d4a5c8455)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.668746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.698058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5858bec4699243b796830abc053b5447'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.705708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5858bec4699243b796830abc053b5447'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.719207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5858bec4699243b796830abc053b5447'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.720019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5858bec4699243b796830abc053b5447'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.720715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5858bec4699243b796830abc053b5447'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.802429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1038/2052] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:37 2022] DELETE /identity/v3/users/5858bec4699243b796830abc053b5447 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:37.803792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:37.805232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:37.871410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce72a82d8ed34f4a962be77d4a5c8455'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.883397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1f368-5b49-4baa-a138-611fe2d8d033 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:37.890388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:37.891178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:37.891529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:37.891885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] PATH_INFO: `/v3/projects/c879759a492b427d99fea0ed4f410908/users/f3b0a65629074281bd0ac256bf4c5ce8/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:37.909544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce72a82d8ed34f4a962be77d4a5c8455'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.910279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c879759a492b427d99fea0ed4f410908, user_id=f3b0a65629074281bd0ac256bf4c5ce8, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:37.915467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:37.922320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce72a82d8ed34f4a962be77d4a5c8455'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.923322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce72a82d8ed34f4a962be77d4a5c8455'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:37.923922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a88e501d-c196-4209-a43e-8b500eb50ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce72a82d8ed34f4a962be77d4a5c8455'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:38.034118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1039/2053] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:37 2022] PUT /identity/v3/projects/c879759a492b427d99fea0ed4f410908/users/f3b0a65629074281bd0ac256bf4c5ce8/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:38.035710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:38.038130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:38.048073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1015/2054] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:37 2022] DELETE /identity/v3/users/ce72a82d8ed34f4a962be77d4a5c8455 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:38.054070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae56a8-9439-4ae7-97fd-afbb3aeb883e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.054598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae56a8-9439-4ae7-97fd-afbb3aeb883e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.055632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae56a8-9439-4ae7-97fd-afbb3aeb883e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.122887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-deae56a8-9439-4ae7-97fd-afbb3aeb883e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:38.126508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-deae56a8-9439-4ae7-97fd-afbb3aeb883e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:38.139596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3944879-6224-46fc-8b61-9818192ebac7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:38.214367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1016/2055] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:38.216194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8741df-3a7c-4310-b5da-ef6bdc19b85e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.216686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8741df-3a7c-4310-b5da-ef6bdc19b85e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.217159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8741df-3a7c-4310-b5da-ef6bdc19b85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.219163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0db47-e602-4532-b4a4-161311e60f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:38.219737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0db47-e602-4532-b4a4-161311e60f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.220020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0db47-e602-4532-b4a4-161311e60f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.220298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0db47-e602-4532-b4a4-161311e60f9d admin admin] PATH_INFO: `/v3/projects/9831c83c9e4a4933a2407061a3b952ac/users/641622fe64c140d4a49bde11d5d2bc62/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.240192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-db8741df-3a7c-4310-b5da-ef6bdc19b85e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:38.240776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0db47-e602-4532-b4a4-161311e60f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9831c83c9e4a4933a2407061a3b952ac, user_id=641622fe64c140d4a49bde11d5d2bc62, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:38.241911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-db8741df-3a7c-4310-b5da-ef6bdc19b85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:38.245665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0db47-e602-4532-b4a4-161311e60f9d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:38.253227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1017/2056] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:38.254527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8741df-3a7c-4310-b5da-ef6bdc19b85e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:38.257963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-db8741df-3a7c-4310-b5da-ef6bdc19b85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:38.330276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8741df-3a7c-4310-b5da-ef6bdc19b85e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:38.333068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1040/2057] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:38 2022] PUT /identity/v3/projects/9831c83c9e4a4933a2407061a3b952ac/users/641622fe64c140d4a49bde11d5d2bc62/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:38.334064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0db47-e602-4532-b4a4-161311e60f9d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:38.394743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925a4b1-87c6-4e7d-95da-13a64d061f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:38.395476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a4b1-87c6-4e7d-95da-13a64d061f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.395877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a4b1-87c6-4e7d-95da-13a64d061f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.396298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a4b1-87c6-4e7d-95da-13a64d061f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.406573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0db47-e602-4532-b4a4-161311e60f9d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:38.410792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:38.412208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.412711 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.413089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.474942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:38.519086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925a4b1-87c6-4e7d-95da-13a64d061f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:38.521221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925a4b1-87c6-4e7d-95da-13a64d061f40 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:38.532909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:38.536552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:38.539067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1018/2058] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:38.541092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a59629-07c5-4b4f-a1b6-fc0ed962648d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.545014 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1041/2059] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:38.545849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:38.545985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a59629-07c5-4b4f-a1b6-fc0ed962648d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.546152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a59629-07c5-4b4f-a1b6-fc0ed962648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.547209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:38.560645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac0620-02e4-44c1-aa35-1a23062b14c4 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:38.566985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:38.569268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.569546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.570059 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] PATH_INFO: `/v3/projects/2863c99428e94f59a20438d518aecde6` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.571356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2863c99428e94f59a20438d518aecde6)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:38.573323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:38.592280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-79a59629-07c5-4b4f-a1b6-fc0ed962648d None None] MFA Rules not processed for user `641622fe64c140d4a49bde11d5d2bc62`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:38.593614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-79a59629-07c5-4b4f-a1b6-fc0ed962648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:38.676094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1019/2060] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:38.681363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a59629-07c5-4b4f-a1b6-fc0ed962648d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:38.692921 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a59629-07c5-4b4f-a1b6-fc0ed962648d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:38.699861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8be7a-54db-4cba-8358-3b961e42e813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:38.700657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8be7a-54db-4cba-8358-3b961e42e813 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.701054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8be7a-54db-4cba-8358-3b961e42e813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.701419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8be7a-54db-4cba-8358-3b961e42e813 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.702160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8be7a-54db-4cba-8358-3b961e42e813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:38.704598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8be7a-54db-4cba-8358-3b961e42e813 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:38.773902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2863c99428e94f59a20438d518aecde6', 'request_id': 'req-473b062a-0694-49a0-a9c3-de6288b80a95'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:38.866599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1020/2061] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:38 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:38.871429 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8be7a-54db-4cba-8358-3b961e42e813 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:38.891691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2863c99428e94f59a20438d518aecde6', 'request_id': 'req-473b062a-0694-49a0-a9c3-de6288b80a95'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:38.906977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2863c99428e94f59a20438d518aecde6', 'request_id': 'req-473b062a-0694-49a0-a9c3-de6288b80a95'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:38.920904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] The token cache is being invalidate because project 2863c99428e94f59a20438d518aecde6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:38.922235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:38.923237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-473b062a-0694-49a0-a9c3-de6288b80a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:38.925301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1042/2062] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:38 2022] DELETE /identity/v3/projects/2863c99428e94f59a20438d518aecde6 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:38.931249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2640d1d-52ce-4eeb-8f6b-e47081c0e838 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.932000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2640d1d-52ce-4eeb-8f6b-e47081c0e838 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.932994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2640d1d-52ce-4eeb-8f6b-e47081c0e838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.982493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d2640d1d-52ce-4eeb-8f6b-e47081c0e838 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:38.984334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d2640d1d-52ce-4eeb-8f6b-e47081c0e838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:38.987280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8be7a-54db-4cba-8358-3b961e42e813 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:38.994178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:38.995096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:38.995332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:38.995644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:38.996310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:38.997676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1043/2063] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:39.001180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.007181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d09d9d-6992-40a4-a5c1-638dcde0ebb8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.007540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d09d9d-6992-40a4-a5c1-638dcde0ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.007890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d09d9d-6992-40a4-a5c1-638dcde0ebb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.017308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:39.035311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-72d09d9d-6992-40a4-a5c1-638dcde0ebb8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:39.036818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-72d09d9d-6992-40a4-a5c1-638dcde0ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:39.048871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1044/2064] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:39.059187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d09d9d-6992-40a4-a5c1-638dcde0ebb8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.072745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d09d9d-6992-40a4-a5c1-638dcde0ebb8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.096853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.096853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.096853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.096853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.100118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:39.100498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.102250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.126093 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1045/2065] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:22:39.126399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.132082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1021/2066] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:38 2022] POST /identity/v3/users => generated 299 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:39.139345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.140587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f16493-65b3-4c74-af78-a1bbfbd1e654 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.141263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:39.156163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.156878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.157220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.157447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.158104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.158350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b25e8-8cd7-4963-9b67-48ad135307c7 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.160075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.163901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cce37f-705d-4618-9849-504c2db97d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.165627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cce37f-705d-4618-9849-504c2db97d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.165627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cce37f-705d-4618-9849-504c2db97d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.165627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cce37f-705d-4618-9849-504c2db97d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.165981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cce37f-705d-4618-9849-504c2db97d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.168988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cce37f-705d-4618-9849-504c2db97d8e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.181853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1022/2067] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:39.188041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cce37f-705d-4618-9849-504c2db97d8e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.202573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cce37f-705d-4618-9849-504c2db97d8e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.210899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c42e3d-bbc6-4d37-90bf-401fe1912a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.211666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42e3d-bbc6-4d37-90bf-401fe1912a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.212062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42e3d-bbc6-4d37-90bf-401fe1912a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.212333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42e3d-bbc6-4d37-90bf-401fe1912a2a admin admin] PATH_INFO: `/v3/projects/b560908d4bae4985bcb94efe45d4e1b2/users/29ced8528096401ba3cac298c08a2bc6/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.229832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c42e3d-bbc6-4d37-90bf-401fe1912a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b560908d4bae4985bcb94efe45d4e1b2, user_id=29ced8528096401ba3cac298c08a2bc6, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.232522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c42e3d-bbc6-4d37-90bf-401fe1912a2a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.296532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1046/2068] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:39 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:39.296532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.298516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:39.360155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1023/2069] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:39 2022] PUT /identity/v3/projects/b560908d4bae4985bcb94efe45d4e1b2/users/29ced8528096401ba3cac298c08a2bc6/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:39.360668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c42e3d-bbc6-4d37-90bf-401fe1912a2a admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.402257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2d1e-0198-4f86-99ab-79cab20c074d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.516231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c42e3d-bbc6-4d37-90bf-401fe1912a2a admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.522527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff478ec-9d29-4d6c-a57c-841c893366dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.523465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff478ec-9d29-4d6c-a57c-841c893366dd admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.523741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff478ec-9d29-4d6c-a57c-841c893366dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.528178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff478ec-9d29-4d6c-a57c-841c893366dd admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.528178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff478ec-9d29-4d6c-a57c-841c893366dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.528178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff478ec-9d29-4d6c-a57c-841c893366dd admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.528749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.529397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.529675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.529995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] PATH_INFO: `/v3/projects/64fc374646b54d7baa9cd77170223cca` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.531293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64fc374646b54d7baa9cd77170223cca)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.533195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.642474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1024/2070] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:39 2022] POST /identity/v3/users => generated 284 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:39.642474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff478ec-9d29-4d6c-a57c-841c893366dd admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.656507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff478ec-9d29-4d6c-a57c-841c893366dd admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.689568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303b8a7-1cf1-40bb-9e3d-c3a97f98169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.690780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303b8a7-1cf1-40bb-9e3d-c3a97f98169c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.691283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303b8a7-1cf1-40bb-9e3d-c3a97f98169c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.691699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303b8a7-1cf1-40bb-9e3d-c3a97f98169c admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.693629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303b8a7-1cf1-40bb-9e3d-c3a97f98169c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.696647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303b8a7-1cf1-40bb-9e3d-c3a97f98169c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.697488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64fc374646b54d7baa9cd77170223cca', 'request_id': 'req-f04609de-c0c1-462d-bdaa-136ae76ea161'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:39.720075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1025/2071] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:39.721448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303b8a7-1cf1-40bb-9e3d-c3a97f98169c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.734739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303b8a7-1cf1-40bb-9e3d-c3a97f98169c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.818311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64fc374646b54d7baa9cd77170223cca', 'request_id': 'req-f04609de-c0c1-462d-bdaa-136ae76ea161'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:39.827673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64fc374646b54d7baa9cd77170223cca', 'request_id': 'req-f04609de-c0c1-462d-bdaa-136ae76ea161'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:39.833386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.834307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.834592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.834861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.835572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.838401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.844184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] The token cache is being invalidate because project 64fc374646b54d7baa9cd77170223cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:39.844794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:39.845627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:39.850847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1047/2072] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:39 2022] DELETE /identity/v3/projects/64fc374646b54d7baa9cd77170223cca => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:39.852441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.854154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:39.855233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1026/2073] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:39.857663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.859723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:39.870553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04609de-c0c1-462d-bdaa-136ae76ea161 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.877567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:39.878322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:39.878603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:39.878879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] PATH_INFO: `/v3/projects/b560908d4bae4985bcb94efe45d4e1b2/users/29ced8528096401ba3cac298c08a2bc6/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:39.881257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b560908d4bae4985bcb94efe45d4e1b2, user_id=29ced8528096401ba3cac298c08a2bc6, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:39.884647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:39.910142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1048/2074] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:39 2022] PUT /identity/v3/projects/b560908d4bae4985bcb94efe45d4e1b2/users/29ced8528096401ba3cac298c08a2bc6/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:39.910142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:39.910142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:39.941539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c78160-363c-4524-aa2f-a902d2876bb3 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.990161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bf410-0c87-4551-864c-81486e3fe304 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:39.999668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:40.004188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:40.004188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:40.004188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] PATH_INFO: `/v3/users/4d76c705da3b4c539c187ab2c45dff22` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:40.004188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d76c705da3b4c539c187ab2c45dff22)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:40.005503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:40.006392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:40.006769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:40.007132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:40.008231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:40.009130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.115269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:40.117710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:40.129231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1027/2075] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:40.132324 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:40.143885 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:40.143885 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.156893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa8b75-38ef-4b63-936f-49d2fe87b535 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:40.163873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083c0ee-bf11-4615-990b-edfa4004f705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:40.164488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083c0ee-bf11-4615-990b-edfa4004f705 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:40.164794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083c0ee-bf11-4615-990b-edfa4004f705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:40.165069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083c0ee-bf11-4615-990b-edfa4004f705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:40.165745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083c0ee-bf11-4615-990b-edfa4004f705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:40.167342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083c0ee-bf11-4615-990b-edfa4004f705 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:40.179639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1028/2076] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:40.180924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43db768-df60-4175-bb4f-035210cbea96 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:40.181212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43db768-df60-4175-bb4f-035210cbea96 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:40.181485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43db768-df60-4175-bb4f-035210cbea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:40.209200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d76c705da3b4c539c187ab2c45dff22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:40.230698 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d76c705da3b4c539c187ab2c45dff22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:40.231931 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a43db768-df60-4175-bb4f-035210cbea96 None None] MFA Rules not processed for user `29ced8528096401ba3cac298c08a2bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:40.233183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a43db768-df60-4175-bb4f-035210cbea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.245603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d76c705da3b4c539c187ab2c45dff22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:40.246502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d76c705da3b4c539c187ab2c45dff22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:40.247618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d76c705da3b4c539c187ab2c45dff22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:40.352310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1029/2077] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:40.352310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43db768-df60-4175-bb4f-035210cbea96 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:40.354112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a43db768-df60-4175-bb4f-035210cbea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.369659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43db768-df60-4175-bb4f-035210cbea96 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:40.378829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:40.380025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:40.380385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:40.380754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] PATH_INFO: `/v3/projects/701c6a7ff3e244cf93b3f7756f6b56a9/users/02394fc2ef2448d8a7e390ab13598c91/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:40.388299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1049/2078] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:39 2022] DELETE /identity/v3/users/4d76c705da3b4c539c187ab2c45dff22 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:40.399217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:40.399217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.426420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=701c6a7ff3e244cf93b3f7756f6b56a9, user_id=02394fc2ef2448d8a7e390ab13598c91, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:40.430373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:40.476325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728664bb-ce2a-4ff5-b21d-036fa4cd90d3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:40.481652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23abc3-d3bb-4b24-a7a3-f8063d806790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:40.482339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23abc3-d3bb-4b24-a7a3-f8063d806790 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:40.483146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23abc3-d3bb-4b24-a7a3-f8063d806790 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:40.483425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23abc3-d3bb-4b24-a7a3-f8063d806790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:40.485494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8d23abc3-d3bb-4b24-a7a3-f8063d806790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.520109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:40.552278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23abc3-d3bb-4b24-a7a3-f8063d806790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:40.553955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23abc3-d3bb-4b24-a7a3-f8063d806790 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:40.563192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1050/2079] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:40.578008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1030/2080] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:40 2022] PUT /identity/v3/projects/701c6a7ff3e244cf93b3f7756f6b56a9/users/02394fc2ef2448d8a7e390ab13598c91/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:40.593202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9656c-3c98-4e72-b192-61b508dddaa3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:40.593510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9656c-3c98-4e72-b192-61b508dddaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:40.593793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9656c-3c98-4e72-b192-61b508dddaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:40.647170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a1d9656c-3c98-4e72-b192-61b508dddaa3 None None] MFA Rules not processed for user `02394fc2ef2448d8a7e390ab13598c91`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:40.649564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a1d9656c-3c98-4e72-b192-61b508dddaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.753512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1051/2081] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:40.768327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:40.773878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.899153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9656c-3c98-4e72-b192-61b508dddaa3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:40.900928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a1d9656c-3c98-4e72-b192-61b508dddaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:40.918070 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0037a8-33a5-474d-9854-96163223af1b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:40.924757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e9154-9854-48c2-acc7-670680be1d06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b5ee9e4388d4bbabc7f1e9e3ca50b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:40.925410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e9154-9854-48c2-acc7-670680be1d06 service glance] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:40.926124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e9154-9854-48c2-acc7-670680be1d06 service glance] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:40.926398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e9154-9854-48c2-acc7-670680be1d06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:40.943458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e9154-9854-48c2-acc7-670680be1d06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:40.948134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e9154-9854-48c2-acc7-670680be1d06 service glance] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:40.958728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1031/2082] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:41.001668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9656c-3c98-4e72-b192-61b508dddaa3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:41.006611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:41.007281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:41.007566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:41.007845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] PATH_INFO: `/v3/projects/eae48b61bbe148a4abcebf212b18d898` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:41.008985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae48b61bbe148a4abcebf212b18d898)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:41.011204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:41.152288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae48b61bbe148a4abcebf212b18d898', 'request_id': 'req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:41.233842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae48b61bbe148a4abcebf212b18d898', 'request_id': 'req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:41.241671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae48b61bbe148a4abcebf212b18d898', 'request_id': 'req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:41.264777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] The token cache is being invalidate because project eae48b61bbe148a4abcebf212b18d898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:41.265065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:41.265818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f82c4a3f-418c-4e8c-8c6e-5ac9868f74aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:41.267994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1052/2083] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:40 2022] DELETE /identity/v3/projects/eae48b61bbe148a4abcebf212b18d898 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:41.274531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97976e-8abb-4d53-9799-d2b4a7900dcf None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:41.274843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97976e-8abb-4d53-9799-d2b4a7900dcf None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:41.275159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97976e-8abb-4d53-9799-d2b4a7900dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:41.297263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-af97976e-8abb-4d53-9799-d2b4a7900dcf None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:41.298498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-af97976e-8abb-4d53-9799-d2b4a7900dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:41.395275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1032/2084] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:41.410315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a69c9-70e3-4be6-ad10-a2b0b6f84c56 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:41.410315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a69c9-70e3-4be6-ad10-a2b0b6f84c56 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:41.410315 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a69c9-70e3-4be6-ad10-a2b0b6f84c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:41.429269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d54a69c9-70e3-4be6-ad10-a2b0b6f84c56 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:41.450629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d54a69c9-70e3-4be6-ad10-a2b0b6f84c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:41.469390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1053/2085] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:41.471131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97976e-8abb-4d53-9799-d2b4a7900dcf None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:41.484309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97976e-8abb-4d53-9799-d2b4a7900dcf None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:41.512697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:41.513494 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:41.513759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:41.514028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:41.514680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:41.514981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:41.516628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:41.550107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1033/2086] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:41.553153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a69c9-70e3-4be6-ad10-a2b0b6f84c56 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:41.587452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a69c9-70e3-4be6-ad10-a2b0b6f84c56 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:41.589645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:41.599518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56674c89-2508-4768-8c7b-93975046c86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:41.601018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56674c89-2508-4768-8c7b-93975046c86e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:41.601608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56674c89-2508-4768-8c7b-93975046c86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:41.602301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56674c89-2508-4768-8c7b-93975046c86e admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:41.603596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56674c89-2508-4768-8c7b-93975046c86e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:41.607936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56674c89-2508-4768-8c7b-93975046c86e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:41.743198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1054/2087] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:41 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:41.746788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:41.758153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d252a-8bd7-4e3a-bf27-182ef73b1576 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:41.835780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89e6c-7ba4-4f8a-be07-7bebdac623ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:41.836382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89e6c-7ba4-4f8a-be07-7bebdac623ad admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:41.836662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89e6c-7ba4-4f8a-be07-7bebdac623ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:41.836993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89e6c-7ba4-4f8a-be07-7bebdac623ad admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:41.837621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc89e6c-7ba4-4f8a-be07-7bebdac623ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:41.839550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc89e6c-7ba4-4f8a-be07-7bebdac623ad admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:41.972594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1034/2088] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:41 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:41.977721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56674c89-2508-4768-8c7b-93975046c86e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:41.997822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56674c89-2508-4768-8c7b-93975046c86e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:42.004275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:42.004941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.005217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.005466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.006120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:42.007748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:42.028263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1055/2089] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:42.031852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89e6c-7ba4-4f8a-be07-7bebdac623ad admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:42.039508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:42.041534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:42.045268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89e6c-7ba4-4f8a-be07-7bebdac623ad admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:42.053530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a2764-333e-4e4f-8f12-1b415dde7522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:42.053530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a2764-333e-4e4f-8f12-1b415dde7522 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.053530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a2764-333e-4e4f-8f12-1b415dde7522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.053530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a2764-333e-4e4f-8f12-1b415dde7522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.053810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a2764-333e-4e4f-8f12-1b415dde7522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:42.053810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a2764-333e-4e4f-8f12-1b415dde7522 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:42.072878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1035/2090] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:42.074009 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a2764-333e-4e4f-8f12-1b415dde7522 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:42.088827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a2764-333e-4e4f-8f12-1b415dde7522 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:42.097144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15494bf0-f2ae-40be-90f1-91e2d5c610a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:42.097820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15494bf0-f2ae-40be-90f1-91e2d5c610a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.098102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15494bf0-f2ae-40be-90f1-91e2d5c610a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.098382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15494bf0-f2ae-40be-90f1-91e2d5c610a4 admin admin] PATH_INFO: `/v3/projects/30badb940864408b942beac2269cd334/users/bc23a15bf3ae4fee953275af262a8078/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.126046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15494bf0-f2ae-40be-90f1-91e2d5c610a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30badb940864408b942beac2269cd334, user_id=bc23a15bf3ae4fee953275af262a8078, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:42.129020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15494bf0-f2ae-40be-90f1-91e2d5c610a4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:42.134872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf820ffe-5151-4bd0-9155-f970fea52f96 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:42.138884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:42.139513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.139799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.140061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.141847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:42.148026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:42.225900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:42.233427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:42.241444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1036/2091] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:42 2022] PUT /identity/v3/projects/30badb940864408b942beac2269cd334/users/bc23a15bf3ae4fee953275af262a8078/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:42.253329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85fecc-7d56-407b-b579-25ac4158f624 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.256811 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85fecc-7d56-407b-b579-25ac4158f624 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.257566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85fecc-7d56-407b-b579-25ac4158f624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.278599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1056/2092] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:42.308113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-5e85fecc-7d56-407b-b579-25ac4158f624 None None] MFA Rules not processed for user `bc23a15bf3ae4fee953275af262a8078`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:42.309532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5e85fecc-7d56-407b-b579-25ac4158f624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:42.420432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1037/2093] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:42.424905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:42.436848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24772c-d45c-4c51-8911-72020d3731e8 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:42.499734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861786c-b85e-4bf5-85a1-62f1856a0d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:42.500365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861786c-b85e-4bf5-85a1-62f1856a0d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.500811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861786c-b85e-4bf5-85a1-62f1856a0d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.501301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861786c-b85e-4bf5-85a1-62f1856a0d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.502153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861786c-b85e-4bf5-85a1-62f1856a0d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:42.507613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861786c-b85e-4bf5-85a1-62f1856a0d8d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:42.587534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1057/2094] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:42 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:42.592266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85fecc-7d56-407b-b579-25ac4158f624 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:42.602658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85fecc-7d56-407b-b579-25ac4158f624 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:42.687874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85183767-bc32-4dc6-806d-f060aaf08f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:42.688969 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85183767-bc32-4dc6-806d-f060aaf08f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.689500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85183767-bc32-4dc6-806d-f060aaf08f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.689927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85183767-bc32-4dc6-806d-f060aaf08f14 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.691170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85183767-bc32-4dc6-806d-f060aaf08f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:42.694333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85183767-bc32-4dc6-806d-f060aaf08f14 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:42.768945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1038/2095] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:42 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:42.770987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861786c-b85e-4bf5-85a1-62f1856a0d8d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:42.781949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861786c-b85e-4bf5-85a1-62f1856a0d8d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:42.787702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ecf5-df00-4d5a-81fa-8b0fcd874cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:42.793253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ecf5-df00-4d5a-81fa-8b0fcd874cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.793253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ecf5-df00-4d5a-81fa-8b0fcd874cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.793253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ecf5-df00-4d5a-81fa-8b0fcd874cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.793253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00ecf5-df00-4d5a-81fa-8b0fcd874cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:42.793253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00ecf5-df00-4d5a-81fa-8b0fcd874cb5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:42.805340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1058/2096] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:42.810234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85183767-bc32-4dc6-806d-f060aaf08f14 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:42.821419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85183767-bc32-4dc6-806d-f060aaf08f14 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:42.826715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b6aac-7a5c-4d28-8a78-25f53a181320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:42.827502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b6aac-7a5c-4d28-8a78-25f53a181320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:42.827788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b6aac-7a5c-4d28-8a78-25f53a181320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:42.828067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b6aac-7a5c-4d28-8a78-25f53a181320 admin admin] PATH_INFO: `/v3/projects/96c974aee24741198993cc002a18d40e/users/ec270b8988d443c491f58875c1362784/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:42.846533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b6aac-7a5c-4d28-8a78-25f53a181320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c974aee24741198993cc002a18d40e, user_id=ec270b8988d443c491f58875c1362784, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:42.850254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b6aac-7a5c-4d28-8a78-25f53a181320 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:42.938690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1039/2097] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:42 2022] PUT /identity/v3/projects/96c974aee24741198993cc002a18d40e/users/ec270b8988d443c491f58875c1362784/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:42.947589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ecf5-df00-4d5a-81fa-8b0fcd874cb5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:42.966089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ecf5-df00-4d5a-81fa-8b0fcd874cb5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:43.044736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa549c8-89ab-46c2-b98e-0fd085eebd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:43.045359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa549c8-89ab-46c2-b98e-0fd085eebd57 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:43.045644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa549c8-89ab-46c2-b98e-0fd085eebd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:43.045923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa549c8-89ab-46c2-b98e-0fd085eebd57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:43.047793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa549c8-89ab-46c2-b98e-0fd085eebd57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:43.048330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa549c8-89ab-46c2-b98e-0fd085eebd57 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:43.063999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b6aac-7a5c-4d28-8a78-25f53a181320 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:43.065192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1059/2098] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:43.077076 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b6aac-7a5c-4d28-8a78-25f53a181320 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:43.083659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:43.084586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:43.085043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:43.085524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] PATH_INFO: `/v3/projects/96c974aee24741198993cc002a18d40e/users/ec270b8988d443c491f58875c1362784/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:43.087738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c974aee24741198993cc002a18d40e, user_id=ec270b8988d443c491f58875c1362784, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:43.090454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:43.102415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1040/2099] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:43 2022] PUT /identity/v3/projects/96c974aee24741198993cc002a18d40e/users/ec270b8988d443c491f58875c1362784/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:43.116630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fcbcc-be16-4c68-b2e5-80f104872598 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:43.116941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fcbcc-be16-4c68-b2e5-80f104872598 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:43.117528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fcbcc-be16-4c68-b2e5-80f104872598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:43.186139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-758fcbcc-be16-4c68-b2e5-80f104872598 None None] MFA Rules not processed for user `ec270b8988d443c491f58875c1362784`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:43.188075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-758fcbcc-be16-4c68-b2e5-80f104872598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:43.286570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1060/2100] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:43.294372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:43.296209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:43.314444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:43.400236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1898-612f-4fce-8441-aa6937d34653 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:43.416419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:43.417101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:43.417400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:43.417720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:43.564732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:43.567356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:43.582335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1041/2101] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:44.422416 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:46.033159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaa675-e959-404e-a6ce-51d2ac464a88 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:46.037173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaa675-e959-404e-a6ce-51d2ac464a88 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:46.037173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaa675-e959-404e-a6ce-51d2ac464a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:46.075291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-36aaa675-e959-404e-a6ce-51d2ac464a88 None None] MFA Rules not processed for user `f3b0a65629074281bd0ac256bf4c5ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:46.076761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-36aaa675-e959-404e-a6ce-51d2ac464a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:46.191699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1061/2102] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:46.195696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:46.197276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:46.301663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21400d32-943f-4edd-a25a-0053fba0e13b service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:46.320107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee500d1-df7f-49ee-b756-af4605a9112d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:46.320742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee500d1-df7f-49ee-b756-af4605a9112d admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:46.324473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee500d1-df7f-49ee-b756-af4605a9112d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:46.324473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee500d1-df7f-49ee-b756-af4605a9112d admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:46.324473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee500d1-df7f-49ee-b756-af4605a9112d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:46.324473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee500d1-df7f-49ee-b756-af4605a9112d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:46.404968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:46.462225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1042/2103] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:46 2022] POST /identity/v3/projects => generated 373 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:46.466463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaa675-e959-404e-a6ce-51d2ac464a88 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:46.482207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaa675-e959-404e-a6ce-51d2ac464a88 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:46.560518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e21ed-5752-44c2-8c7f-43c6ab07baf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:46.561257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e21ed-5752-44c2-8c7f-43c6ab07baf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:46.561577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e21ed-5752-44c2-8c7f-43c6ab07baf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:46.561802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e21ed-5752-44c2-8c7f-43c6ab07baf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:46.562469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e21ed-5752-44c2-8c7f-43c6ab07baf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:46.565172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e21ed-5752-44c2-8c7f-43c6ab07baf3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:46.673570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1062/2104] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:46 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:46.679834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee500d1-df7f-49ee-b756-af4605a9112d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:46.709712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee500d1-df7f-49ee-b756-af4605a9112d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:46.716191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1cf8d-44ec-4893-9f7e-7fbd1825db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:46.717116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1cf8d-44ec-4893-9f7e-7fbd1825db5c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:46.717400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1cf8d-44ec-4893-9f7e-7fbd1825db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:46.717905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1cf8d-44ec-4893-9f7e-7fbd1825db5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:46.718695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1cf8d-44ec-4893-9f7e-7fbd1825db5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:46.720358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1cf8d-44ec-4893-9f7e-7fbd1825db5c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:46.734435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1043/2105] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:22:46.738914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e21ed-5752-44c2-8c7f-43c6ab07baf3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:46.751251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e21ed-5752-44c2-8c7f-43c6ab07baf3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:46.756331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf130-5ecc-486e-906c-3c5ab853872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:46.757147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf130-5ecc-486e-906c-3c5ab853872f admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:46.757453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf130-5ecc-486e-906c-3c5ab853872f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:46.757728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf130-5ecc-486e-906c-3c5ab853872f admin admin] PATH_INFO: `/v3/projects/2a41158c1e664ece8158abc22df41a8c/users/f7b7421aa5a84af89e2ac77de7589333/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:46.773715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf130-5ecc-486e-906c-3c5ab853872f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a41158c1e664ece8158abc22df41a8c, user_id=f7b7421aa5a84af89e2ac77de7589333, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:46.785683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf130-5ecc-486e-906c-3c5ab853872f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:46.866000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1063/2106] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:46 2022] PUT /identity/v3/projects/2a41158c1e664ece8158abc22df41a8c/users/f7b7421aa5a84af89e2ac77de7589333/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:22:46.870380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1cf8d-44ec-4893-9f7e-7fbd1825db5c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:46.887272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1cf8d-44ec-4893-9f7e-7fbd1825db5c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:46.975390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbbbb4-0299-4de6-a6a1-aa10ee5bbc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:46.976118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbbbb4-0299-4de6-a6a1-aa10ee5bbc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:46.978526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbbbb4-0299-4de6-a6a1-aa10ee5bbc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:46.978526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbbbb4-0299-4de6-a6a1-aa10ee5bbc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:46.978526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbbbb4-0299-4de6-a6a1-aa10ee5bbc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:46.979708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbbbb4-0299-4de6-a6a1-aa10ee5bbc46 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:46.994265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1044/2107] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:46.997879 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf130-5ecc-486e-906c-3c5ab853872f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:47.008342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf130-5ecc-486e-906c-3c5ab853872f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:47.013345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2190ce-9749-4241-9b46-d87c770a13b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:47.016590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2190ce-9749-4241-9b46-d87c770a13b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:47.016590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2190ce-9749-4241-9b46-d87c770a13b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:47.016590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2190ce-9749-4241-9b46-d87c770a13b0 admin admin] PATH_INFO: `/v3/projects/2a41158c1e664ece8158abc22df41a8c/users/f7b7421aa5a84af89e2ac77de7589333/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:47.016590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2190ce-9749-4241-9b46-d87c770a13b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a41158c1e664ece8158abc22df41a8c, user_id=f7b7421aa5a84af89e2ac77de7589333, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:47.019304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2190ce-9749-4241-9b46-d87c770a13b0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:47.035068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1064/2108] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:46 2022] PUT /identity/v3/projects/2a41158c1e664ece8158abc22df41a8c/users/f7b7421aa5a84af89e2ac77de7589333/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:47.141481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:47.889899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:47.976632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:48.946534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:49.426965 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:49.806143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:50.485783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:50.709308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:50.775567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:50.858666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbbbb4-0299-4de6-a6a1-aa10ee5bbc46 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:50.958959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbbbb4-0299-4de6-a6a1-aa10ee5bbc46 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:51.059807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:51.060511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:51.060798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:51.061072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:51.155102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2190ce-9749-4241-9b46-d87c770a13b0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:51.168139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2190ce-9749-4241-9b46-d87c770a13b0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:51.168712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:51.170415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:51.172452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f8823-0a63-4490-9f27-1cbe0c4b8765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:51.173330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f8823-0a63-4490-9f27-1cbe0c4b8765 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:51.173763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f8823-0a63-4490-9f27-1cbe0c4b8765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:51.174239 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f8823-0a63-4490-9f27-1cbe0c4b8765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:51.176544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-355f8823-0a63-4490-9f27-1cbe0c4b8765 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:51.217987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1045/2109] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:51.262897 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f8823-0a63-4490-9f27-1cbe0c4b8765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:51.264872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f8823-0a63-4490-9f27-1cbe0c4b8765 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:51.291468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1065/2110] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:22:51.531801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:51.632872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:51.895571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:52.318761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:52.320972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:52.402164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907620e-5f37-4545-b19b-3c60998e1d9e service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:52.407476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:52.408107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:52.408388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:52.408829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] PATH_INFO: `/v3/users/444a1ac1fc82456c98e18ae6eb961ce8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:52.409965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=444a1ac1fc82456c98e18ae6eb961ce8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:52.412021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:52.478661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:52.646903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444a1ac1fc82456c98e18ae6eb961ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:52.665147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444a1ac1fc82456c98e18ae6eb961ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:52.674195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444a1ac1fc82456c98e18ae6eb961ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:52.675291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444a1ac1fc82456c98e18ae6eb961ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:52.676231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444a1ac1fc82456c98e18ae6eb961ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:52.761660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1046/2111] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:52 2022] DELETE /identity/v3/users/444a1ac1fc82456c98e18ae6eb961ce8 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:52.767226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e6c8-69c7-4e17-8378-dba4000b87b3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:52.768947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e6c8-69c7-4e17-8378-dba4000b87b3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:52.769487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e6c8-69c7-4e17-8378-dba4000b87b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:52.797487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9b84e6c8-69c7-4e17-8378-dba4000b87b3 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:52.798795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9b84e6c8-69c7-4e17-8378-dba4000b87b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:52.872290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1066/2112] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:52.882876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:52.885602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:52.886305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:52.972137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1539bea-acbd-42d8-b192-c04b846f3238 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:52.978725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec188e6-ecfb-48d6-bb43-dcf0eef575da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:52.979886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec188e6-ecfb-48d6-bb43-dcf0eef575da service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:52.980167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec188e6-ecfb-48d6-bb43-dcf0eef575da service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:52.983253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec188e6-ecfb-48d6-bb43-dcf0eef575da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:52.996575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec188e6-ecfb-48d6-bb43-dcf0eef575da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:52.996910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec188e6-ecfb-48d6-bb43-dcf0eef575da service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:53.028767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1047/2113] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:53.240566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:53.558316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84e6c8-69c7-4e17-8378-dba4000b87b3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:53.560654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9b84e6c8-69c7-4e17-8378-dba4000b87b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:53.582398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84e6c8-69c7-4e17-8378-dba4000b87b3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:53.605189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:53.606322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:53.606826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:53.607402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] PATH_INFO: `/v3/projects/549f902c93c0487e957156b6744b5f1a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:53.609298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=549f902c93c0487e957156b6744b5f1a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:53.611903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:53.724046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '549f902c93c0487e957156b6744b5f1a', 'request_id': 'req-6ed71cf0-1bb4-4904-8632-0d383428c4d1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:53.805306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '549f902c93c0487e957156b6744b5f1a', 'request_id': 'req-6ed71cf0-1bb4-4904-8632-0d383428c4d1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:53.812151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '549f902c93c0487e957156b6744b5f1a', 'request_id': 'req-6ed71cf0-1bb4-4904-8632-0d383428c4d1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:53.837569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:53.843583 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] The token cache is being invalidate because project 549f902c93c0487e957156b6744b5f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:53.844139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:53.845371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6ed71cf0-1bb4-4904-8632-0d383428c4d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:53.847446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1067/2114] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:53 2022] DELETE /identity/v3/projects/549f902c93c0487e957156b6744b5f1a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:53.851509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec188e6-ecfb-48d6-bb43-dcf0eef575da service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:53.865109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec188e6-ecfb-48d6-bb43-dcf0eef575da service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:53.951765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:53.952688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:53.953127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:53.953552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] PATH_INFO: `/v3/users/208372b2aeeb4deea2fa6b74a752b2b5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:53.955971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] RBAC: Authorizing `identity:delete_user(user_id=208372b2aeeb4deea2fa6b74a752b2b5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:53.957478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:53.980331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:54.190591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208372b2aeeb4deea2fa6b74a752b2b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:54.203740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208372b2aeeb4deea2fa6b74a752b2b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:54.211431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208372b2aeeb4deea2fa6b74a752b2b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:54.212261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208372b2aeeb4deea2fa6b74a752b2b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:54.212918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208372b2aeeb4deea2fa6b74a752b2b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:54.310452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1048/2115] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:53 2022] DELETE /identity/v3/users/208372b2aeeb4deea2fa6b74a752b2b5 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:54.643499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:54.807255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94cf02-df6d-4f62-9060-b51f4e54bc3c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:54.808192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94cf02-df6d-4f62-9060-b51f4e54bc3c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:54.808459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94cf02-df6d-4f62-9060-b51f4e54bc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:54.840579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fe94cf02-df6d-4f62-9060-b51f4e54bc3c None None] MFA Rules not processed for user `f7b7421aa5a84af89e2ac77de7589333`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:54.841837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fe94cf02-df6d-4f62-9060-b51f4e54bc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:54.911429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1068/2116] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:54.924795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:54.927090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:54.969861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94cf02-df6d-4f62-9060-b51f4e54bc3c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:54.971634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fe94cf02-df6d-4f62-9060-b51f4e54bc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:55.043527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393a225-4fb2-4955-b08e-892397f5403b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:55.051923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:55.052858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:55.053627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:55.054124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:55.057170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:55.064025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94cf02-df6d-4f62-9060-b51f4e54bc3c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:55.075184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:55.076005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:55.076605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:55.076894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] PATH_INFO: `/v3/projects/1c1a79ce70b0486aa19a7aca35b9c835` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:55.079941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c1a79ce70b0486aa19a7aca35b9c835)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:55.081926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:55.140601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:55.143188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:55.153663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1049/2117] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:55.197144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c1a79ce70b0486aa19a7aca35b9c835', 'request_id': 'req-777e51b1-9584-4206-87b3-c94d03db35e3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.207513 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:55.320133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c1a79ce70b0486aa19a7aca35b9c835', 'request_id': 'req-777e51b1-9584-4206-87b3-c94d03db35e3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.326837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c1a79ce70b0486aa19a7aca35b9c835', 'request_id': 'req-777e51b1-9584-4206-87b3-c94d03db35e3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.340140 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] The token cache is being invalidate because project 1c1a79ce70b0486aa19a7aca35b9c835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:55.340632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.341363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.343045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1069/2118] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:54 2022] DELETE /identity/v3/projects/1c1a79ce70b0486aa19a7aca35b9c835 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:55.346970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:55.359188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61aee9-c336-49fc-a3ee-30ad859b6346 service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:55.444000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:55.444809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:55.445228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:55.445425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] PATH_INFO: `/v3/users/0def5ff84c724cf4be406945b747990e` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:55.447382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0def5ff84c724cf4be406945b747990e)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:55.449774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:55.699660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0def5ff84c724cf4be406945b747990e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.723554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0def5ff84c724cf4be406945b747990e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.732028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0def5ff84c724cf4be406945b747990e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.733067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0def5ff84c724cf4be406945b747990e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.733882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d0ada6b7-08b6-4941-a7d8-d7c80b9f5fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0def5ff84c724cf4be406945b747990e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:55.824886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1050/2119] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:22:55 2022] DELETE /identity/v3/users/0def5ff84c724cf4be406945b747990e => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:55.843667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:56.444567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:56.446776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:56.481624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e51b1-9584-4206-87b3-c94d03db35e3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:56.639915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:56.640935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:56.641400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:56.641850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] PATH_INFO: `/v3/projects/dce1a4765a9941889af243b4f61675e7` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:56.643715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce1a4765a9941889af243b4f61675e7)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:56.647708 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:56.693782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:56.829988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce1a4765a9941889af243b4f61675e7', 'request_id': 'req-735afaec-60c7-4cfb-95c6-c19a42a837c5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:56.911220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce1a4765a9941889af243b4f61675e7', 'request_id': 'req-735afaec-60c7-4cfb-95c6-c19a42a837c5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:56.935117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce1a4765a9941889af243b4f61675e7', 'request_id': 'req-735afaec-60c7-4cfb-95c6-c19a42a837c5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:56.964066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] The token cache is being invalidate because project dce1a4765a9941889af243b4f61675e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:22:56.966497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:56.968862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-735afaec-60c7-4cfb-95c6-c19a42a837c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:22:56.973885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1070/2120] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:22:56 2022] DELETE /identity/v3/projects/dce1a4765a9941889af243b4f61675e7 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:56.985197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db881573-b433-4619-ba14-c84b7f6bcf42 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:56.985197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db881573-b433-4619-ba14-c84b7f6bcf42 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:56.985197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db881573-b433-4619-ba14-c84b7f6bcf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.002965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-db881573-b433-4619-ba14-c84b7f6bcf42 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:57.004250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-db881573-b433-4619-ba14-c84b7f6bcf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:57.070182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1051/2121] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:57.080882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515eb2-b4ec-4326-a6a0-c40139c41e48 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:57.081273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515eb2-b4ec-4326-a6a0-c40139c41e48 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:57.081561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515eb2-b4ec-4326-a6a0-c40139c41e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.105084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c4515eb2-b4ec-4326-a6a0-c40139c41e48 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:57.106420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c4515eb2-b4ec-4326-a6a0-c40139c41e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:57.118789 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1071/2122] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:57.123170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db881573-b433-4619-ba14-c84b7f6bcf42 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:57.140811 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db881573-b433-4619-ba14-c84b7f6bcf42 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:57.144726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:57.145395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:57.145669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:57.146040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.146691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:22:57.146992 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:57.148627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:57.167106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1052/2123] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:22:57.168373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4515eb2-b4ec-4326-a6a0-c40139c41e48 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:57.181145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4515eb2-b4ec-4326-a6a0-c40139c41e48 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:57.187271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec8378-61a1-4c51-8d3f-9c90588549bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:57.188007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec8378-61a1-4c51-8d3f-9c90588549bb admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:57.189200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec8378-61a1-4c51-8d3f-9c90588549bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:57.189554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec8378-61a1-4c51-8d3f-9c90588549bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.191761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec8378-61a1-4c51-8d3f-9c90588549bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:57.194633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec8378-61a1-4c51-8d3f-9c90588549bb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:57.407441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1072/2124] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:22:57 2022] POST /identity/v3/projects => generated 381 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:22:57.412113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:57.423842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaf3ed-79b4-4e24-9c70-0ca2b4b9b49d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:57.516271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3ed71-9e44-4b71-923e-e424d0ac81db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:57.516919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ed71-9e44-4b71-923e-e424d0ac81db admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:57.517212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ed71-9e44-4b71-923e-e424d0ac81db admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:57.517483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ed71-9e44-4b71-923e-e424d0ac81db admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.518693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3ed71-9e44-4b71-923e-e424d0ac81db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:57.521240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3ed71-9e44-4b71-923e-e424d0ac81db admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:57.616246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1053/2125] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:22:57 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:22:57.621030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec8378-61a1-4c51-8d3f-9c90588549bb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:57.633432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec8378-61a1-4c51-8d3f-9c90588549bb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:57.639336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833fe5-8f2e-489f-a63b-a19e7df0c540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:57.642535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833fe5-8f2e-489f-a63b-a19e7df0c540 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:57.642535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833fe5-8f2e-489f-a63b-a19e7df0c540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:57.642535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833fe5-8f2e-489f-a63b-a19e7df0c540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.647167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a833fe5-8f2e-489f-a63b-a19e7df0c540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:57.649248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a833fe5-8f2e-489f-a63b-a19e7df0c540 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:57.662411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1073/2126] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:57.666490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3ed71-9e44-4b71-923e-e424d0ac81db admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:57.677756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3ed71-9e44-4b71-923e-e424d0ac81db admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:57.683167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cb4cb-6306-40e6-9f60-b92e38a3e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:57.683870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb4cb-6306-40e6-9f60-b92e38a3e074 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:57.684218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb4cb-6306-40e6-9f60-b92e38a3e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:57.684563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb4cb-6306-40e6-9f60-b92e38a3e074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.685289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cb4cb-6306-40e6-9f60-b92e38a3e074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:57.686981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cb4cb-6306-40e6-9f60-b92e38a3e074 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:57.718296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833fe5-8f2e-489f-a63b-a19e7df0c540 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:57.719149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1054/2127] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:22:57.733107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833fe5-8f2e-489f-a63b-a19e7df0c540 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:57.739518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:57.740274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:57.740892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:57.741545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] PATH_INFO: `/v3/projects/7bdb62eaaade46ffa205828d13dba7be/users/b68f7224855a4cec81b664a2573d60f1/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.768685 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bdb62eaaade46ffa205828d13dba7be, user_id=b68f7224855a4cec81b664a2573d60f1, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:57.771604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:57.876024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:57.883824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1074/2128] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:22:57 2022] PUT /identity/v3/projects/7bdb62eaaade46ffa205828d13dba7be/users/b68f7224855a4cec81b664a2573d60f1/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:22:57.889066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80017b5-4a85-466a-ace6-c2117a46d08a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:57.889614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80017b5-4a85-466a-ace6-c2117a46d08a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:57.890143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80017b5-4a85-466a-ace6-c2117a46d08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:57.919920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f80017b5-4a85-466a-ace6-c2117a46d08a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:57.921643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f80017b5-4a85-466a-ace6-c2117a46d08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:58.036703 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1055/2129] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:58.046617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:58.050646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:58.143193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ce082-643e-4c90-a1c7-30f12107c5ba admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:22:58.147859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0c572-1532-4f84-83ea-702814da67c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:22:58.148464 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0c572-1532-4f84-83ea-702814da67c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:58.148744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0c572-1532-4f84-83ea-702814da67c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:58.149193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0c572-1532-4f84-83ea-702814da67c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:58.158823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0c572-1532-4f84-83ea-702814da67c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:22:58.160377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0c572-1532-4f84-83ea-702814da67c2 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:22:58.171787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1075/2130] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:22:58.490844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:22:58.706601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06d63-3d1e-42be-ab7c-4184d30e562e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:58.706888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06d63-3d1e-42be-ab7c-4184d30e562e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:58.707222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06d63-3d1e-42be-ab7c-4184d30e562e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:58.737233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d7a06d63-3d1e-42be-ab7c-4184d30e562e None None] MFA Rules not processed for user `f3b0a65629074281bd0ac256bf4c5ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:58.738485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d7a06d63-3d1e-42be-ab7c-4184d30e562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:58.796377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1056/2131] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:22:59.811417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3efac-e3f6-4390-bf4b-a2971c76baa9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:22:59.811953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3efac-e3f6-4390-bf4b-a2971c76baa9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:22:59.811953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3efac-e3f6-4390-bf4b-a2971c76baa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:22:59.860734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-02e3efac-e3f6-4390-bf4b-a2971c76baa9 None None] MFA Rules not processed for user `b68f7224855a4cec81b664a2573d60f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:22:59.860734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-02e3efac-e3f6-4390-bf4b-a2971c76baa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:22:59.931375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1076/2132] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:22:59.960293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a06d63-3d1e-42be-ab7c-4184d30e562e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:22:59.960293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d7a06d63-3d1e-42be-ab7c-4184d30e562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:00.012959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a06d63-3d1e-42be-ab7c-4184d30e562e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:00.016827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:00.017438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:00.017894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:00.018174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:00.027704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:00.029190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:00.038069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1057/2133] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:00.085009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:00.599614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:01.022549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:01.376616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:01.523829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:01.609437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:01.770882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:01.804755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:01.945999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:01.975157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.060444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.146256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.176601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.347677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.376339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.433285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.461815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.570848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.587752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.741362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.811099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.883645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:02.987692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:03.193821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:03.313523 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:03.491612 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:03.662834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:03.866684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:03.936567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fb9fe-a3be-4273-b888-7e8a24f34f9d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:03.936838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fb9fe-a3be-4273-b888-7e8a24f34f9d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:03.937111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fb9fe-a3be-4273-b888-7e8a24f34f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:03.956958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ec2fb9fe-a3be-4273-b888-7e8a24f34f9d None None] MFA Rules not processed for user `b68f7224855a4cec81b664a2573d60f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:03.958925 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ec2fb9fe-a3be-4273-b888-7e8a24f34f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:03.965984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1077/2134] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:04.470245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:04.471971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:04.486376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebd16d-2bb3-45ac-94a9-c46543500f7a service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:04.491784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:04.492452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:04.492736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:04.493010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] PATH_INFO: `/v3/users/641622fe64c140d4a49bde11d5d2bc62` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:04.494149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=641622fe64c140d4a49bde11d5d2bc62)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:04.496196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:04.691467 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '641622fe64c140d4a49bde11d5d2bc62'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:04.704398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '641622fe64c140d4a49bde11d5d2bc62'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:04.711074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '641622fe64c140d4a49bde11d5d2bc62'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:04.711756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '641622fe64c140d4a49bde11d5d2bc62'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:04.712364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '641622fe64c140d4a49bde11d5d2bc62'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:04.821994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1058/2135] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:04 2022] DELETE /identity/v3/users/641622fe64c140d4a49bde11d5d2bc62 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:04.826428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a391a3-0ae3-4344-af75-660c8069b170 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:04.826959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a391a3-0ae3-4344-af75-660c8069b170 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:04.826959 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a391a3-0ae3-4344-af75-660c8069b170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:04.845899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-90a391a3-0ae3-4344-af75-660c8069b170 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:04.847157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-90a391a3-0ae3-4344-af75-660c8069b170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:04.928226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1078/2136] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:04.938583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:04.940049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:05.002455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d6765-0317-47c4-9539-46aa784fdef6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:05.006932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ec091-424b-44a3-b74e-cc6573ad8ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:05.007582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ec091-424b-44a3-b74e-cc6573ad8ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:05.007863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ec091-424b-44a3-b74e-cc6573ad8ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:05.008137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ec091-424b-44a3-b74e-cc6573ad8ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:05.019270 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ec091-424b-44a3-b74e-cc6573ad8ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:05.021560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ec091-424b-44a3-b74e-cc6573ad8ea8 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:05.037563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1059/2137] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:23:05.710164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a391a3-0ae3-4344-af75-660c8069b170 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:05.712729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-90a391a3-0ae3-4344-af75-660c8069b170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:05.732283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a391a3-0ae3-4344-af75-660c8069b170 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:05.738639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:05.740168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:05.740617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:05.740617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] PATH_INFO: `/v3/projects/9831c83c9e4a4933a2407061a3b952ac` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:05.747728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=9831c83c9e4a4933a2407061a3b952ac)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:05.750141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:05.878047 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9831c83c9e4a4933a2407061a3b952ac', 'request_id': 'req-881e0542-98f9-4ee4-8263-dc72271a8cef'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:05.964963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9831c83c9e4a4933a2407061a3b952ac', 'request_id': 'req-881e0542-98f9-4ee4-8263-dc72271a8cef'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:05.974222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9831c83c9e4a4933a2407061a3b952ac', 'request_id': 'req-881e0542-98f9-4ee4-8263-dc72271a8cef'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:05.991177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] The token cache is being invalidate because project 9831c83c9e4a4933a2407061a3b952ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:05.991573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:05.992287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:05.993777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1079/2138] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:05 2022] DELETE /identity/v3/projects/9831c83c9e4a4933a2407061a3b952ac => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:05.997845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ec091-424b-44a3-b74e-cc6573ad8ea8 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:06.008400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ec091-424b-44a3-b74e-cc6573ad8ea8 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:06.073902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:06.074527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:06.074803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:06.075105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] PATH_INFO: `/v3/users/29ced8528096401ba3cac298c08a2bc6` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:06.076323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ced8528096401ba3cac298c08a2bc6)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:06.078157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:06.089245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:06.295265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ced8528096401ba3cac298c08a2bc6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:06.308701 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ced8528096401ba3cac298c08a2bc6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:06.316206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ced8528096401ba3cac298c08a2bc6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:06.316957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ced8528096401ba3cac298c08a2bc6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:06.317988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-faa3bd82-f56b-4b0b-873a-4071d0ef4d95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ced8528096401ba3cac298c08a2bc6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:06.413521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1060/2139] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:05 2022] DELETE /identity/v3/users/29ced8528096401ba3cac298c08a2bc6 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:06.920457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:06.922561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:06.948235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e0542-98f9-4ee4-8263-dc72271a8cef admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:07.027316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:07.034608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:07.035445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:07.036342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:07.036896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] PATH_INFO: `/v3/projects/b560908d4bae4985bcb94efe45d4e1b2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:07.038249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b560908d4bae4985bcb94efe45d4e1b2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:07.040367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:07.261263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b560908d4bae4985bcb94efe45d4e1b2', 'request_id': 'req-b1702905-c34f-45a8-b871-041d0d57fc1d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:07.336931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b560908d4bae4985bcb94efe45d4e1b2', 'request_id': 'req-b1702905-c34f-45a8-b871-041d0d57fc1d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:07.349853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b560908d4bae4985bcb94efe45d4e1b2', 'request_id': 'req-b1702905-c34f-45a8-b871-041d0d57fc1d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:07.363475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] The token cache is being invalidate because project b560908d4bae4985bcb94efe45d4e1b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:07.364164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:07.364920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b1702905-c34f-45a8-b871-041d0d57fc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:07.366857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1080/2140] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:06 2022] DELETE /identity/v3/projects/b560908d4bae4985bcb94efe45d4e1b2 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:07.372648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642220b-8e08-402f-a5e3-5ecf5171efda None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:07.373116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642220b-8e08-402f-a5e3-5ecf5171efda None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:07.373197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642220b-8e08-402f-a5e3-5ecf5171efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:07.407527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1642220b-8e08-402f-a5e3-5ecf5171efda None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:07.408799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1642220b-8e08-402f-a5e3-5ecf5171efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:07.495402 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1061/2141] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:07.504595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475f8f-bcce-4e37-b2e9-dea58f71d1a0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:07.504901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475f8f-bcce-4e37-b2e9-dea58f71d1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:07.505187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475f8f-bcce-4e37-b2e9-dea58f71d1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:07.536860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a8475f8f-bcce-4e37-b2e9-dea58f71d1a0 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:07.538510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a8475f8f-bcce-4e37-b2e9-dea58f71d1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:07.548899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1081/2142] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:23:07.553873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642220b-8e08-402f-a5e3-5ecf5171efda None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:07.571766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642220b-8e08-402f-a5e3-5ecf5171efda None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:07.577173 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:07.577859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:07.578135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:07.578608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:07.579300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:23:07.579608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:07.581223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:07.594409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1062/2143] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:23:07.599085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8475f8f-bcce-4e37-b2e9-dea58f71d1a0 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:07.608427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8475f8f-bcce-4e37-b2e9-dea58f71d1a0 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:07.613097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb79161-d226-4c97-ba14-e7c825876773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:07.613690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb79161-d226-4c97-ba14-e7c825876773 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:07.614903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb79161-d226-4c97-ba14-e7c825876773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:07.615221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb79161-d226-4c97-ba14-e7c825876773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:07.615847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb79161-d226-4c97-ba14-e7c825876773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:07.617745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb79161-d226-4c97-ba14-e7c825876773 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:07.722263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1082/2144] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:07 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:07.729840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:07.744881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a372d83-f03a-4a9f-85c2-9a8ef617e24c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:07.816222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dbafc-111e-4e53-8c8d-0262ae0607f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:07.817141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dbafc-111e-4e53-8c8d-0262ae0607f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:07.817600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dbafc-111e-4e53-8c8d-0262ae0607f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:07.818038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dbafc-111e-4e53-8c8d-0262ae0607f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:07.818837 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dbafc-111e-4e53-8c8d-0262ae0607f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:07.820983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dbafc-111e-4e53-8c8d-0262ae0607f4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:07.914797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1063/2145] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:07 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:07.918482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb79161-d226-4c97-ba14-e7c825876773 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:07.931533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb79161-d226-4c97-ba14-e7c825876773 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:07.940173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112eea1f-94ca-4c2f-9623-a61f080af3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:07.940803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eea1f-94ca-4c2f-9623-a61f080af3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:07.941416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eea1f-94ca-4c2f-9623-a61f080af3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:07.941944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eea1f-94ca-4c2f-9623-a61f080af3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:07.942845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eea1f-94ca-4c2f-9623-a61f080af3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:07.944785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eea1f-94ca-4c2f-9623-a61f080af3ef admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:07.957316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1083/2146] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:07.961957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dbafc-111e-4e53-8c8d-0262ae0607f4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:07.969293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:07.974563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dbafc-111e-4e53-8c8d-0262ae0607f4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:07.979978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfadd3-80fd-4b18-a758-f17da37a5f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:07.980695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfadd3-80fd-4b18-a758-f17da37a5f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:07.981056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfadd3-80fd-4b18-a758-f17da37a5f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:07.981549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfadd3-80fd-4b18-a758-f17da37a5f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:07.984157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfadd3-80fd-4b18-a758-f17da37a5f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:07.984157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfadd3-80fd-4b18-a758-f17da37a5f40 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:07.998481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1064/2147] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:08.003857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112eea1f-94ca-4c2f-9623-a61f080af3ef admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:08.015071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112eea1f-94ca-4c2f-9623-a61f080af3ef admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:08.021607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:08.022356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:08.022740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:08.023158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] PATH_INFO: `/v3/projects/a8b0cbeb78734b7cbf62d144e980c1c9/users/6cc4329a0d00458b93025d32054326a6/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:08.043732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b0cbeb78734b7cbf62d144e980c1c9, user_id=6cc4329a0d00458b93025d32054326a6, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:08.047291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:08.129343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1084/2148] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:08 2022] PUT /identity/v3/projects/a8b0cbeb78734b7cbf62d144e980c1c9/users/6cc4329a0d00458b93025d32054326a6/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:08.142649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54941b-da24-405f-8fe8-a8f3bd7be0c3 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:08.142938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54941b-da24-405f-8fe8-a8f3bd7be0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:08.143296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54941b-da24-405f-8fe8-a8f3bd7be0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:08.172920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8f54941b-da24-405f-8fe8-a8f3bd7be0c3 None None] MFA Rules not processed for user `6cc4329a0d00458b93025d32054326a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:08.174177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8f54941b-da24-405f-8fe8-a8f3bd7be0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:08.245297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1065/2149] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:23:08.262743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:08.262743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:08.341349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87aede2-e8cf-4635-bb16-92302863a61e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:08.346548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2d0b3-4414-4450-9544-2ab357a6074f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:08.347277 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d0b3-4414-4450-9544-2ab357a6074f service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:08.347559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d0b3-4414-4450-9544-2ab357a6074f service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:08.347830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d0b3-4414-4450-9544-2ab357a6074f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:08.365616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2d0b3-4414-4450-9544-2ab357a6074f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:08.367076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2d0b3-4414-4450-9544-2ab357a6074f service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:08.383609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1085/2150] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:09.588984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d4c2-bfae-4ae5-b3e2-72682bf27e56 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:09.589449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d4c2-bfae-4ae5-b3e2-72682bf27e56 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:09.589534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749d4c2-bfae-4ae5-b3e2-72682bf27e56 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:09.591135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1066/2151] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:23:09.806200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3d9e7-5759-4a3f-ab4c-6d0d7b3e27e7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:09.806908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3d9e7-5759-4a3f-ab4c-6d0d7b3e27e7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:09.806908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3d9e7-5759-4a3f-ab4c-6d0d7b3e27e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:09.830534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4df3d9e7-5759-4a3f-ab4c-6d0d7b3e27e7 None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:09.832202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4df3d9e7-5759-4a3f-ab4c-6d0d7b3e27e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:09.856406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:09.912991 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1086/2152] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:09.923928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749d4c2-bfae-4ae5-b3e2-72682bf27e56 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:09.925608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7749d4c2-bfae-4ae5-b3e2-72682bf27e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:09.943457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749d4c2-bfae-4ae5-b3e2-72682bf27e56 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:09.948623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ac737-5af7-4b04-96ba-bb088c1227d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:09.949220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ac737-5af7-4b04-96ba-bb088c1227d3 service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:09.949499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ac737-5af7-4b04-96ba-bb088c1227d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:09.949767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ac737-5af7-4b04-96ba-bb088c1227d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:09.958191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ac737-5af7-4b04-96ba-bb088c1227d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:09.959806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ac737-5af7-4b04-96ba-bb088c1227d3 service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:09.969377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1067/2153] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:09.983964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3d9e7-5759-4a3f-ab4c-6d0d7b3e27e7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:09.985761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4df3d9e7-5759-4a3f-ab4c-6d0d7b3e27e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:10.065124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3d9e7-5759-4a3f-ab4c-6d0d7b3e27e7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:10.069878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:10.070584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:10.070858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:10.071166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] PATH_INFO: `/v3/users/5401baffbd794016b01d109e8e2529ac` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:10.072312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5401baffbd794016b01d109e8e2529ac)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:10.074191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:10.280179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5401baffbd794016b01d109e8e2529ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.287785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5401baffbd794016b01d109e8e2529ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.299700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5401baffbd794016b01d109e8e2529ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.300459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5401baffbd794016b01d109e8e2529ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.301062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9f0d262b-d3f4-45dd-96d2-8233071087f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5401baffbd794016b01d109e8e2529ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.410475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1087/2154] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:09 2022] DELETE /identity/v3/users/5401baffbd794016b01d109e8e2529ac => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:10.697600 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ac737-5af7-4b04-96ba-bb088c1227d3 service placement] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:10.708604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ac737-5af7-4b04-96ba-bb088c1227d3 service placement] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:10.773567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:10.774199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:10.774480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:10.774985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] PATH_INFO: `/v3/projects/5becdfe9ba3b48d5b11668d6eeec33a4` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:10.776046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] RBAC: Authorizing `identity:delete_project(project_id=5becdfe9ba3b48d5b11668d6eeec33a4)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:10.778038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:10.879738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5becdfe9ba3b48d5b11668d6eeec33a4', 'request_id': 'req-5e8fe6e5-ea8e-429f-94e5-a4090826daab'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.964690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5becdfe9ba3b48d5b11668d6eeec33a4', 'request_id': 'req-5e8fe6e5-ea8e-429f-94e5-a4090826daab'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.971210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5becdfe9ba3b48d5b11668d6eeec33a4', 'request_id': 'req-5e8fe6e5-ea8e-429f-94e5-a4090826daab'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.979795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:10.983597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] The token cache is being invalidate because project 5becdfe9ba3b48d5b11668d6eeec33a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:10.983886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.984619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e8fe6e5-ea8e-429f-94e5-a4090826daab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:10.986229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1068/2155] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:10 2022] DELETE /identity/v3/projects/5becdfe9ba3b48d5b11668d6eeec33a4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:10.992082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1483f-b339-4642-bd72-04073f9a3268 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:10.992574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1483f-b339-4642-bd72-04073f9a3268 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:10.992688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1483f-b339-4642-bd72-04073f9a3268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.012762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b9e1483f-b339-4642-bd72-04073f9a3268 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:11.013985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b9e1483f-b339-4642-bd72-04073f9a3268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:11.050404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:11.075509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1088/2156] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:11.083742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432d98f-e55f-4204-be82-4b63f4dea1bd None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.084030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432d98f-e55f-4204-be82-4b63f4dea1bd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.084306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432d98f-e55f-4204-be82-4b63f4dea1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.108401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1432d98f-e55f-4204-be82-4b63f4dea1bd None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:11.108401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1432d98f-e55f-4204-be82-4b63f4dea1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:11.118075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1069/2157] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:11.122476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1483f-b339-4642-bd72-04073f9a3268 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:11.132566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1483f-b339-4642-bd72-04073f9a3268 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:11.138210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:11.138843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.139166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.139442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.140146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:23:11.140449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:11.144833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:11.154573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1089/2158] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:23:11.158804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432d98f-e55f-4204-be82-4b63f4dea1bd None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:11.168988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432d98f-e55f-4204-be82-4b63f4dea1bd None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:11.173956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e9074-8ae3-4973-82d0-094d9dc700c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:11.174565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e9074-8ae3-4973-82d0-094d9dc700c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.174850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e9074-8ae3-4973-82d0-094d9dc700c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.175300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e9074-8ae3-4973-82d0-094d9dc700c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.175951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e9074-8ae3-4973-82d0-094d9dc700c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:11.178264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e9074-8ae3-4973-82d0-094d9dc700c6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:11.321383 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1070/2159] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:11 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:11.325522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:11.347196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3be9c-0ef2-44b8-a51f-680671e7502f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:11.440856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cdc7da-6b8a-4fe8-8d9d-c354d1d67339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:11.443386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdc7da-6b8a-4fe8-8d9d-c354d1d67339 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.443771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdc7da-6b8a-4fe8-8d9d-c354d1d67339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.444265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdc7da-6b8a-4fe8-8d9d-c354d1d67339 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.445543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cdc7da-6b8a-4fe8-8d9d-c354d1d67339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:11.448203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cdc7da-6b8a-4fe8-8d9d-c354d1d67339 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:11.537305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1090/2160] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:11 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:11.538626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e9074-8ae3-4973-82d0-094d9dc700c6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:11.548958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e9074-8ae3-4973-82d0-094d9dc700c6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:11.554657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e37b5-7540-4634-973f-d6471ec7cd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:11.555314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e37b5-7540-4634-973f-d6471ec7cd6f admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.555751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e37b5-7540-4634-973f-d6471ec7cd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.556025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e37b5-7540-4634-973f-d6471ec7cd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.556677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e37b5-7540-4634-973f-d6471ec7cd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:11.558305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e37b5-7540-4634-973f-d6471ec7cd6f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:11.572788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1071/2161] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:11.574014 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cdc7da-6b8a-4fe8-8d9d-c354d1d67339 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:11.585593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cdc7da-6b8a-4fe8-8d9d-c354d1d67339 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:11.590505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87deed62-a505-4047-94a1-eb2cf356980d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:11.591219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deed62-a505-4047-94a1-eb2cf356980d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.591700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deed62-a505-4047-94a1-eb2cf356980d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.592051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deed62-a505-4047-94a1-eb2cf356980d admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.592776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87deed62-a505-4047-94a1-eb2cf356980d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:11.594508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87deed62-a505-4047-94a1-eb2cf356980d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:11.619053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1091/2162] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:11.621906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e37b5-7540-4634-973f-d6471ec7cd6f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:11.654367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e37b5-7540-4634-973f-d6471ec7cd6f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:11.664058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:11.665344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.665344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.665901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] PATH_INFO: `/v3/projects/555fce0b106b485db6a6800bdced08d2/users/10eaef565cf54b939bf78eba060f83d0/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.684271 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=555fce0b106b485db6a6800bdced08d2, user_id=10eaef565cf54b939bf78eba060f83d0, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:11.686905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:11.759352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1072/2163] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:11 2022] PUT /identity/v3/projects/555fce0b106b485db6a6800bdced08d2/users/10eaef565cf54b939bf78eba060f83d0/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:11.768710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e84f4-d15e-4e75-b89f-bbab33f650dc None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.768998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e84f4-d15e-4e75-b89f-bbab33f650dc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.769278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e84f4-d15e-4e75-b89f-bbab33f650dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.789368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-244e84f4-d15e-4e75-b89f-bbab33f650dc None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:11.790579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-244e84f4-d15e-4e75-b89f-bbab33f650dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:11.849361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1092/2164] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:11.859560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:11.861501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:11.923852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f35c4a-ca1a-4581-b6bc-e2e38f6107b7 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:11.927943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:11.928581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:11.928952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:11.929254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:11.939407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:11.940897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:11.950819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1073/2165] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:17.837039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc312-9447-48b4-8ab1-ddac6a322f1a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:17.837039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc312-9447-48b4-8ab1-ddac6a322f1a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:17.837039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc312-9447-48b4-8ab1-ddac6a322f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:17.877413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-649dc312-9447-48b4-8ab1-ddac6a322f1a None None] MFA Rules not processed for user `10eaef565cf54b939bf78eba060f83d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:17.878662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-649dc312-9447-48b4-8ab1-ddac6a322f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:17.924992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:17.979194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1093/2166] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:17.988852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:17.993781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:18.118744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6074f-6a0a-4eb6-9069-dda9c3df837e service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:18.125427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:18.126049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:18.126335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:18.126614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:18.144074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:18.144463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:18.156121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1074/2167] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:21.537466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dba02-aecf-4f11-9033-a561857ec71f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:21.538465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dba02-aecf-4f11-9033-a561857ec71f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:21.538749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dba02-aecf-4f11-9033-a561857ec71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:21.567749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-dc6dba02-aecf-4f11-9033-a561857ec71f None None] MFA Rules not processed for user `10eaef565cf54b939bf78eba060f83d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:21.568955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dc6dba02-aecf-4f11-9033-a561857ec71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:21.580108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1094/2168] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:21.854234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:21.869013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:21.885940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f149d59-7e39-4909-a217-e13236d71d3a service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:21.892163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:21.893141 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:21.893657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:21.894119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] PATH_INFO: `/v3/users/6cc4329a0d00458b93025d32054326a6` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:21.895790 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc4329a0d00458b93025d32054326a6)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:21.898265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:21.928823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:22.182668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc4329a0d00458b93025d32054326a6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:22.208748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc4329a0d00458b93025d32054326a6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:22.216569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc4329a0d00458b93025d32054326a6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:22.217360 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc4329a0d00458b93025d32054326a6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:22.218563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc4329a0d00458b93025d32054326a6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:22.301678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1075/2169] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:21 2022] DELETE /identity/v3/users/6cc4329a0d00458b93025d32054326a6 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:22.303528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cfb3-07cb-49de-a453-51691ecd4edf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:22.304076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cfb3-07cb-49de-a453-51691ecd4edf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:22.304556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cfb3-07cb-49de-a453-51691ecd4edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:22.329378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9f16cfb3-07cb-49de-a453-51691ecd4edf None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:22.330749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9f16cfb3-07cb-49de-a453-51691ecd4edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:22.406800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1095/2170] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:22.417858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:22.419938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:22.522415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c6ce-8764-4551-9e32-e06942a20070 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:22.539256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c52887-0f31-457d-bb45-3810d7263db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:22.539718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52887-0f31-457d-bb45-3810d7263db2 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:22.540345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52887-0f31-457d-bb45-3810d7263db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:22.540630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52887-0f31-457d-bb45-3810d7263db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:22.559226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52887-0f31-457d-bb45-3810d7263db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:22.563948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52887-0f31-457d-bb45-3810d7263db2 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:22.585928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1076/2171] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:23.294039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16cfb3-07cb-49de-a453-51691ecd4edf None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:23.299292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9f16cfb3-07cb-49de-a453-51691ecd4edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:23.314581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16cfb3-07cb-49de-a453-51691ecd4edf None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:23.320818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:23.321637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:23.321919 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:23.322630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] PATH_INFO: `/v3/projects/a8b0cbeb78734b7cbf62d144e980c1c9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:23.324865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8b0cbeb78734b7cbf62d144e980c1c9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:23.328139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:23.496537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b0cbeb78734b7cbf62d144e980c1c9', 'request_id': 'req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:23.558209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b0cbeb78734b7cbf62d144e980c1c9', 'request_id': 'req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:23.573257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b0cbeb78734b7cbf62d144e980c1c9', 'request_id': 'req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:23.586673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] The token cache is being invalidate because project a8b0cbeb78734b7cbf62d144e980c1c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:23.589229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:23.589229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ce4812ca-4456-4d7b-af09-32eaa21ca0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:23.592582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1096/2172] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:23 2022] DELETE /identity/v3/projects/a8b0cbeb78734b7cbf62d144e980c1c9 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:23.594870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00442c-0b4a-49f3-bf29-fe91e72a3d60 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:23.594870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00442c-0b4a-49f3-bf29-fe91e72a3d60 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:23.594870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00442c-0b4a-49f3-bf29-fe91e72a3d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:23.622017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-fa00442c-0b4a-49f3-bf29-fe91e72a3d60 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:23.623973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-fa00442c-0b4a-49f3-bf29-fe91e72a3d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:23.714995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1077/2173] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:23.723129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbf63c-41be-4039-85d0-b56a187f249c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:23.732890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbf63c-41be-4039-85d0-b56a187f249c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:23.732890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbf63c-41be-4039-85d0-b56a187f249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:23.745196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-21fbf63c-41be-4039-85d0-b56a187f249c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:23.746407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-21fbf63c-41be-4039-85d0-b56a187f249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:23.755587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1097/2174] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:23.761579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00442c-0b4a-49f3-bf29-fe91e72a3d60 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:23.769811 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00442c-0b4a-49f3-bf29-fe91e72a3d60 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:23.774622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:23.775259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:23.775546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:23.775821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:23.776519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:23:23.776820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:23.778422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:23.794093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1078/2175] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:23:23.795223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbf63c-41be-4039-85d0-b56a187f249c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:23.805055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbf63c-41be-4039-85d0-b56a187f249c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:23.809677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d636a2-9977-4d22-988c-7d495ea59db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:23.810265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d636a2-9977-4d22-988c-7d495ea59db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:23.810542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d636a2-9977-4d22-988c-7d495ea59db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:23.810812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d636a2-9977-4d22-988c-7d495ea59db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:23.811459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d636a2-9977-4d22-988c-7d495ea59db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:23.813339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d636a2-9977-4d22-988c-7d495ea59db2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:23.927083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1098/2176] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:23 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:23.931374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:23.941947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe9b92-9f96-40eb-ad43-b1147d93fe35 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.001392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d2bab-eb1a-47be-bd14-38dc0d9f2ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.002003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d2bab-eb1a-47be-bd14-38dc0d9f2ded admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.002286 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d2bab-eb1a-47be-bd14-38dc0d9f2ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.002562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d2bab-eb1a-47be-bd14-38dc0d9f2ded admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.003239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d2bab-eb1a-47be-bd14-38dc0d9f2ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.005825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d2bab-eb1a-47be-bd14-38dc0d9f2ded admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.084993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1079/2177] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:23 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:24.089133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d636a2-9977-4d22-988c-7d495ea59db2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.099934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d636a2-9977-4d22-988c-7d495ea59db2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.102179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:24.104644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1caa-85cb-452d-9c53-ce36e3221f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.105244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1caa-85cb-452d-9c53-ce36e3221f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.105526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1caa-85cb-452d-9c53-ce36e3221f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.105794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1caa-85cb-452d-9c53-ce36e3221f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.106439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da1caa-85cb-452d-9c53-ce36e3221f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.108092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da1caa-85cb-452d-9c53-ce36e3221f61 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.118781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1099/2178] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:24.128063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d2bab-eb1a-47be-bd14-38dc0d9f2ded admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.138855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d2bab-eb1a-47be-bd14-38dc0d9f2ded admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.143387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee4c5a-e321-40cb-8ad3-102002f02ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.143976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee4c5a-e321-40cb-8ad3-102002f02ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.144254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee4c5a-e321-40cb-8ad3-102002f02ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.144523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee4c5a-e321-40cb-8ad3-102002f02ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.145246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee4c5a-e321-40cb-8ad3-102002f02ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.147283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee4c5a-e321-40cb-8ad3-102002f02ad5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.157747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1080/2179] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:24.162529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1caa-85cb-452d-9c53-ce36e3221f61 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.172444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1caa-85cb-452d-9c53-ce36e3221f61 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.177522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.178201 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.178481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.178755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] PATH_INFO: `/v3/projects/2d0d271d466a4b40ae02a5a89b76b71f/users/6fafd14760f84b3fa41ffd97929a91b4/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.190356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:24.194939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0d271d466a4b40ae02a5a89b76b71f, user_id=6fafd14760f84b3fa41ffd97929a91b4, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.197694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.285529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1100/2180] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:24 2022] PUT /identity/v3/projects/2d0d271d466a4b40ae02a5a89b76b71f/users/6fafd14760f84b3fa41ffd97929a91b4/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:24.296328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db020-16f2-47c1-b015-b1d9619cb05e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.296639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db020-16f2-47c1-b015-b1d9619cb05e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.296914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db020-16f2-47c1-b015-b1d9619cb05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.325744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-639db020-16f2-47c1-b015-b1d9619cb05e None None] MFA Rules not processed for user `6fafd14760f84b3fa41ffd97929a91b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:24.326943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-639db020-16f2-47c1-b015-b1d9619cb05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:24.381103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.382784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:24.408592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1081/2181] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:24.415112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639db020-16f2-47c1-b015-b1d9619cb05e None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.487187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639db020-16f2-47c1-b015-b1d9619cb05e None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.493060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773544f3-11e5-4790-ae15-42410fea739c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.498097 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519dd0e1-ba37-4ca4-9698-eeb2f8907000 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.498210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773544f3-11e5-4790-ae15-42410fea739c admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.498210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773544f3-11e5-4790-ae15-42410fea739c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.498210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773544f3-11e5-4790-ae15-42410fea739c admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.498210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773544f3-11e5-4790-ae15-42410fea739c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.498210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773544f3-11e5-4790-ae15-42410fea739c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.498609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.498818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.499378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.499818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] PATH_INFO: `/v3/users/bc23a15bf3ae4fee953275af262a8078` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.501314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc23a15bf3ae4fee953275af262a8078)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.504048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.581910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1082/2182] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:24 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:23:24.585996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773544f3-11e5-4790-ae15-42410fea739c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.646903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773544f3-11e5-4790-ae15-42410fea739c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.651940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.652570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.652852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.653126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.653753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.655970 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.675777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc23a15bf3ae4fee953275af262a8078'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:24.683390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc23a15bf3ae4fee953275af262a8078'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:24.695857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc23a15bf3ae4fee953275af262a8078'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:24.696546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc23a15bf3ae4fee953275af262a8078'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:24.697174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8b317011-37ac-4dc2-99ad-ea965ad30524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc23a15bf3ae4fee953275af262a8078'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:24.730488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1083/2183] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:24 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:23:24.734505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.736236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:24.747223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1101/2184] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:24 2022] DELETE /identity/v3/users/bc23a15bf3ae4fee953275af262a8078 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:24.751219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d3f1a-04ff-49f4-aa72-6eedcdde3065 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.752123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658e3b4-b0ac-4871-8c32-0eeb99797deb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.752413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658e3b4-b0ac-4871-8c32-0eeb99797deb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.752690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658e3b4-b0ac-4871-8c32-0eeb99797deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.774082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5658e3b4-b0ac-4871-8c32-0eeb99797deb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:24.775375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5658e3b4-b0ac-4871-8c32-0eeb99797deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:24.821492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c7c9-3d87-48ee-91a9-0644061ad702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.822082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c7c9-3d87-48ee-91a9-0644061ad702 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.822368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c7c9-3d87-48ee-91a9-0644061ad702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.822671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c7c9-3d87-48ee-91a9-0644061ad702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.823496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22c7c9-3d87-48ee-91a9-0644061ad702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.825101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22c7c9-3d87-48ee-91a9-0644061ad702 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.835444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1084/2185] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:23:24.839367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c7c9-3d87-48ee-91a9-0644061ad702 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.846341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1102/2186] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:24.849596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c7c9-3d87-48ee-91a9-0644061ad702 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.854331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0457e-cc42-4586-b097-c9f472a16ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.854926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0457e-cc42-4586-b097-c9f472a16ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.855244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0457e-cc42-4586-b097-c9f472a16ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.855526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0457e-cc42-4586-b097-c9f472a16ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.856169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0457e-cc42-4586-b097-c9f472a16ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.856377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658e3b4-b0ac-4871-8c32-0eeb99797deb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.857718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5658e3b4-b0ac-4871-8c32-0eeb99797deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:24.857835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0457e-cc42-4586-b097-c9f472a16ae1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.868720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1085/2187] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:23:24.872938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0457e-cc42-4586-b097-c9f472a16ae1 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:24.883129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0457e-cc42-4586-b097-c9f472a16ae1 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.888122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11881bc-a568-4f79-8294-bd95b7c745a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.888772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11881bc-a568-4f79-8294-bd95b7c745a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.889059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11881bc-a568-4f79-8294-bd95b7c745a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.889337 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11881bc-a568-4f79-8294-bd95b7c745a8 admin admin] PATH_INFO: `/v3/projects/1d7a9ec0e541495bb9f15650b3b1b8b1/users/bec50d15fbae42cba748b23dd3769a87/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.904475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11881bc-a568-4f79-8294-bd95b7c745a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7a9ec0e541495bb9f15650b3b1b8b1, user_id=bec50d15fbae42cba748b23dd3769a87, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.907244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11881bc-a568-4f79-8294-bd95b7c745a8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.915378 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658e3b4-b0ac-4871-8c32-0eeb99797deb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:24.919280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbabc6c-2a3f-407f-a05c-6c767b106d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:24.919884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbabc6c-2a3f-407f-a05c-6c767b106d7b service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.920166 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbabc6c-2a3f-407f-a05c-6c767b106d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.920454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbabc6c-2a3f-407f-a05c-6c767b106d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:24.929578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbabc6c-2a3f-407f-a05c-6c767b106d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:24.930989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbabc6c-2a3f-407f-a05c-6c767b106d7b service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:24.940212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1103/2188] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:23:24.975089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1086/2189] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:24 2022] PUT /identity/v3/projects/1d7a9ec0e541495bb9f15650b3b1b8b1/users/bec50d15fbae42cba748b23dd3769a87/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:24.986279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331b95-64de-4720-b5f4-bbcd8b2cb809 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:24.986566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331b95-64de-4720-b5f4-bbcd8b2cb809 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:24.986846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331b95-64de-4720-b5f4-bbcd8b2cb809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.015377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-88331b95-64de-4720-b5f4-bbcd8b2cb809 None None] MFA Rules not processed for user `bec50d15fbae42cba748b23dd3769a87`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:25.016601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-88331b95-64de-4720-b5f4-bbcd8b2cb809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:25.082262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1104/2190] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:25.085753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11881bc-a568-4f79-8294-bd95b7c745a8 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.142497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:25.172748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11881bc-a568-4f79-8294-bd95b7c745a8 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:25.193926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:25.194590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.194870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.195232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.195888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:25.198263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:25.274294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:25.310369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1087/2191] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:25 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:25.314428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88331b95-64de-4720-b5f4-bbcd8b2cb809 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.325872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88331b95-64de-4720-b5f4-bbcd8b2cb809 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:25.368419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.371561 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:25.394516 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d5baa-8bd7-4681-ba55-1950bc2289fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:25.395256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d5baa-8bd7-4681-ba55-1950bc2289fa admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.395543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d5baa-8bd7-4681-ba55-1950bc2289fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.395820 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d5baa-8bd7-4681-ba55-1950bc2289fa admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.396468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d5baa-8bd7-4681-ba55-1950bc2289fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:25.398696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d5baa-8bd7-4681-ba55-1950bc2289fa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:25.458904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdbd8b-5cf9-4c33-96bd-7b39bcc5352c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:25.463739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:25.464371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.464658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.464942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] PATH_INFO: `/v3/projects/30badb940864408b942beac2269cd334` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.466106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30badb940864408b942beac2269cd334)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:25.468133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:25.483365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1105/2192] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:25 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:25.491156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d5baa-8bd7-4681-ba55-1950bc2289fa admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.501819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d5baa-8bd7-4681-ba55-1950bc2289fa admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:25.509585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34e33-ef35-4369-9bc0-9382b7d846a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:25.511663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34e33-ef35-4369-9bc0-9382b7d846a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.512082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34e33-ef35-4369-9bc0-9382b7d846a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.512365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34e33-ef35-4369-9bc0-9382b7d846a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.513033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f34e33-ef35-4369-9bc0-9382b7d846a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:25.514656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f34e33-ef35-4369-9bc0-9382b7d846a6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:25.525192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1106/2193] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:25.529512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34e33-ef35-4369-9bc0-9382b7d846a6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.539824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34e33-ef35-4369-9bc0-9382b7d846a6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:25.544647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f7ad-ca8f-43db-b173-d9863ac99ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:25.545290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f7ad-ca8f-43db-b173-d9863ac99ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.545570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f7ad-ca8f-43db-b173-d9863ac99ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.545843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f7ad-ca8f-43db-b173-d9863ac99ff3 admin admin] PATH_INFO: `/v3/projects/88abd9f8a61e4d77a78e86bd8b4e38c2/users/2f129cfede2a47859fd76b9deda1e1df/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.562892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38f7ad-ca8f-43db-b173-d9863ac99ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88abd9f8a61e4d77a78e86bd8b4e38c2, user_id=2f129cfede2a47859fd76b9deda1e1df, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:25.565610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38f7ad-ca8f-43db-b173-d9863ac99ff3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:25.667278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1107/2194] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:25 2022] PUT /identity/v3/projects/88abd9f8a61e4d77a78e86bd8b4e38c2/users/2f129cfede2a47859fd76b9deda1e1df/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:25.671150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f7ad-ca8f-43db-b173-d9863ac99ff3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.675253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30badb940864408b942beac2269cd334', 'request_id': 'req-614999a3-90cd-426c-94d1-a8230e66f8c9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:25.731890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f7ad-ca8f-43db-b173-d9863ac99ff3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:25.751866 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30badb940864408b942beac2269cd334', 'request_id': 'req-614999a3-90cd-426c-94d1-a8230e66f8c9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:25.758077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30badb940864408b942beac2269cd334', 'request_id': 'req-614999a3-90cd-426c-94d1-a8230e66f8c9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:25.769635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] The token cache is being invalidate because project 30badb940864408b942beac2269cd334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:25.770012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:25.770773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:25.772468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1088/2195] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:25 2022] DELETE /identity/v3/projects/30badb940864408b942beac2269cd334 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:25.776367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.777682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:25.794832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:25.795469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.795748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.796023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.796666 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:25.798342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:25.809081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1108/2196] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:23:25.813596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.814843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:25.827299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2d6c9-2605-47e2-bec5-4b06fac8af2a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:25.833017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df27bf-c6a7-44ee-b2fb-3d140033d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:25.833017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df27bf-c6a7-44ee-b2fb-3d140033d153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.833017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df27bf-c6a7-44ee-b2fb-3d140033d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.833329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df27bf-c6a7-44ee-b2fb-3d140033d153 admin admin] PATH_INFO: `/v3/projects/88abd9f8a61e4d77a78e86bd8b4e38c2/users/2f129cfede2a47859fd76b9deda1e1df/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.833411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:25.835410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df27bf-c6a7-44ee-b2fb-3d140033d153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88abd9f8a61e4d77a78e86bd8b4e38c2, user_id=2f129cfede2a47859fd76b9deda1e1df, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:25.838038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df27bf-c6a7-44ee-b2fb-3d140033d153 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:25.846041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614999a3-90cd-426c-94d1-a8230e66f8c9 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:25.849293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1109/2197] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:25 2022] PUT /identity/v3/projects/88abd9f8a61e4d77a78e86bd8b4e38c2/users/2f129cfede2a47859fd76b9deda1e1df/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:25.859342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7331f-07ec-4a55-9f59-a447c1a10d6f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.859622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7331f-07ec-4a55-9f59-a447c1a10d6f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.859899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7331f-07ec-4a55-9f59-a447c1a10d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.890768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a7c7331f-07ec-4a55-9f59-a447c1a10d6f None None] MFA Rules not processed for user `2f129cfede2a47859fd76b9deda1e1df`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:25.892024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a7c7331f-07ec-4a55-9f59-a447c1a10d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:25.910345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:25.911823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:25.912031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:25.912306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] PATH_INFO: `/v3/users/ec270b8988d443c491f58875c1362784` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:25.913478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec270b8988d443c491f58875c1362784)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:25.915464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:25.965222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1110/2198] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:23:25.976608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7331f-07ec-4a55-9f59-a447c1a10d6f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:25.977963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a7c7331f-07ec-4a55-9f59-a447c1a10d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:26.032889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7331f-07ec-4a55-9f59-a447c1a10d6f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:26.036576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:26.037173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:26.037447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:26.037715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:26.039477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:26.079850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:26.099755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:26.101225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:26.110280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1111/2199] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:23:26.127649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec270b8988d443c491f58875c1362784'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:26.141240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:26.142329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec270b8988d443c491f58875c1362784'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:26.150401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec270b8988d443c491f58875c1362784'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:26.151178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec270b8988d443c491f58875c1362784'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:26.151834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-068c162c-97a2-431c-be4c-0f6e8ca7cff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec270b8988d443c491f58875c1362784'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:26.249613 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1089/2200] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:25 2022] DELETE /identity/v3/users/ec270b8988d443c491f58875c1362784 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:26.582735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:26.766686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:26.777860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa1595-8f47-4702-ba56-c36ff8162ca5 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:26.843787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:26.844500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:26.844852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:26.845322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] PATH_INFO: `/v3/projects/96c974aee24741198993cc002a18d40e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:26.846512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c974aee24741198993cc002a18d40e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:26.848557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:26.955979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c974aee24741198993cc002a18d40e', 'request_id': 'req-46037f1f-d114-42e1-ac93-09e143987b44'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:27.061759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c974aee24741198993cc002a18d40e', 'request_id': 'req-46037f1f-d114-42e1-ac93-09e143987b44'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:27.068439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c974aee24741198993cc002a18d40e', 'request_id': 'req-46037f1f-d114-42e1-ac93-09e143987b44'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:27.080021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] The token cache is being invalidate because project 96c974aee24741198993cc002a18d40e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:27.080310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:27.080964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-46037f1f-d114-42e1-ac93-09e143987b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:27.082450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1112/2201] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:26 2022] DELETE /identity/v3/projects/96c974aee24741198993cc002a18d40e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:27.087310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98118d4-05ea-4b89-999a-61158107a5d0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.087778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98118d4-05ea-4b89-999a-61158107a5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.087885 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98118d4-05ea-4b89-999a-61158107a5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.107787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e98118d4-05ea-4b89-999a-61158107a5d0 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:27.109003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e98118d4-05ea-4b89-999a-61158107a5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:27.118124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:27.170227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1090/2202] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:27.178453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d26cc-ac30-4d04-9060-559df5a20f7f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.178749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d26cc-ac30-4d04-9060-559df5a20f7f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.179055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d26cc-ac30-4d04-9060-559df5a20f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.198602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b05d26cc-ac30-4d04-9060-559df5a20f7f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:27.199882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b05d26cc-ac30-4d04-9060-559df5a20f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:27.208163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1113/2203] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:27.212441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98118d4-05ea-4b89-999a-61158107a5d0 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:27.222414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98118d4-05ea-4b89-999a-61158107a5d0 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:27.227740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:27.228344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.228623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.228897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.230779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:23:27.230779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:27.231470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:27.243719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1091/2204] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:23:27.248190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d26cc-ac30-4d04-9060-559df5a20f7f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:27.257727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d26cc-ac30-4d04-9060-559df5a20f7f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:27.262623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb29bb2-92d5-42d2-9f01-637346921166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:27.263252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb29bb2-92d5-42d2-9f01-637346921166 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.263533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb29bb2-92d5-42d2-9f01-637346921166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.263802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb29bb2-92d5-42d2-9f01-637346921166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.264422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb29bb2-92d5-42d2-9f01-637346921166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:27.266383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb29bb2-92d5-42d2-9f01-637346921166 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:27.401312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1114/2205] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:27 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:27.405729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:27.416224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd056d1-41d2-4af6-a5ab-73c317fd8f24 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:27.457231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:27.477525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eba06-6f85-4e17-8cdf-7bdd8503cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:27.478194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eba06-6f85-4e17-8cdf-7bdd8503cceb admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.478469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eba06-6f85-4e17-8cdf-7bdd8503cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.478739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eba06-6f85-4e17-8cdf-7bdd8503cceb admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.479425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196eba06-6f85-4e17-8cdf-7bdd8503cceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:27.481504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196eba06-6f85-4e17-8cdf-7bdd8503cceb admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:27.577682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1092/2206] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:27 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:27.583881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb29bb2-92d5-42d2-9f01-637346921166 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:27.603163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb29bb2-92d5-42d2-9f01-637346921166 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:27.608350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77f9cd-5355-4b57-a470-e7a1175835af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:27.608981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77f9cd-5355-4b57-a470-e7a1175835af admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.609261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77f9cd-5355-4b57-a470-e7a1175835af admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.609534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77f9cd-5355-4b57-a470-e7a1175835af admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.610376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77f9cd-5355-4b57-a470-e7a1175835af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:27.612174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77f9cd-5355-4b57-a470-e7a1175835af admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:27.628348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1115/2207] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:23:27.629582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eba06-6f85-4e17-8cdf-7bdd8503cceb admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:27.643806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eba06-6f85-4e17-8cdf-7bdd8503cceb admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:27.650056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68005376-283f-47a6-b532-71d02173e7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:27.651136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68005376-283f-47a6-b532-71d02173e7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.651476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68005376-283f-47a6-b532-71d02173e7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.651828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68005376-283f-47a6-b532-71d02173e7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.652804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68005376-283f-47a6-b532-71d02173e7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:27.659519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68005376-283f-47a6-b532-71d02173e7d9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:27.681380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1093/2208] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:27.687230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77f9cd-5355-4b57-a470-e7a1175835af admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:27.702493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77f9cd-5355-4b57-a470-e7a1175835af admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:27.712081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b66ee7-6c09-420a-8a45-40ab53a58855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:27.712928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66ee7-6c09-420a-8a45-40ab53a58855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.713327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66ee7-6c09-420a-8a45-40ab53a58855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.713776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66ee7-6c09-420a-8a45-40ab53a58855 admin admin] PATH_INFO: `/v3/projects/8e6f37ea0d454f37b39b21f3dc73290a/users/4a02fafc3b004b51959d344e4c1bb24f/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.737721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b66ee7-6c09-420a-8a45-40ab53a58855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6f37ea0d454f37b39b21f3dc73290a, user_id=4a02fafc3b004b51959d344e4c1bb24f, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:27.740676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b66ee7-6c09-420a-8a45-40ab53a58855 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:27.856979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1116/2209] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:27 2022] PUT /identity/v3/projects/8e6f37ea0d454f37b39b21f3dc73290a/users/4a02fafc3b004b51959d344e4c1bb24f/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:27.868054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c8de3-23b4-497c-94c7-79569526ff40 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:27.868336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c8de3-23b4-497c-94c7-79569526ff40 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:27.868728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c8de3-23b4-497c-94c7-79569526ff40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:27.913207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c58c8de3-23b4-497c-94c7-79569526ff40 None None] MFA Rules not processed for user `4a02fafc3b004b51959d344e4c1bb24f`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:27.914770 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c58c8de3-23b4-497c-94c7-79569526ff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:28.010178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1094/2210] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:28.015367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b66ee7-6c09-420a-8a45-40ab53a58855 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:28.088201 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b66ee7-6c09-420a-8a45-40ab53a58855 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:28.093222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eaa57d-60ba-4d51-b80b-5649d42642d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:28.093941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaa57d-60ba-4d51-b80b-5649d42642d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:28.094900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaa57d-60ba-4d51-b80b-5649d42642d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:28.094900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaa57d-60ba-4d51-b80b-5649d42642d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:28.095672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eaa57d-60ba-4d51-b80b-5649d42642d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:28.097709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eaa57d-60ba-4d51-b80b-5649d42642d7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:28.203126 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1117/2211] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:28 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:28.205321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c8de3-23b4-497c-94c7-79569526ff40 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:28.216553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c8de3-23b4-497c-94c7-79569526ff40 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:28.279331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12be5-abb1-4212-841c-edfc43d9bbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:28.279945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12be5-abb1-4212-841c-edfc43d9bbcb admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:28.280228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12be5-abb1-4212-841c-edfc43d9bbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:28.280502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12be5-abb1-4212-841c-edfc43d9bbcb admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:28.281138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d12be5-abb1-4212-841c-edfc43d9bbcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:28.283094 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d12be5-abb1-4212-841c-edfc43d9bbcb admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:28.379032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1095/2212] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:28 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:28.382339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eaa57d-60ba-4d51-b80b-5649d42642d7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:28.392033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eaa57d-60ba-4d51-b80b-5649d42642d7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:28.396623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a49f-2fdc-4eb8-90fc-cb6be86aa986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:28.397858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a49f-2fdc-4eb8-90fc-cb6be86aa986 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:28.398130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a49f-2fdc-4eb8-90fc-cb6be86aa986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:28.398406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a49f-2fdc-4eb8-90fc-cb6be86aa986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:28.399074 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199a49f-2fdc-4eb8-90fc-cb6be86aa986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:28.400682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199a49f-2fdc-4eb8-90fc-cb6be86aa986 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:28.412032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1118/2213] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:23:28.416122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12be5-abb1-4212-841c-edfc43d9bbcb admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:28.425968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12be5-abb1-4212-841c-edfc43d9bbcb admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:28.431300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aae015-6b7d-4248-9257-8dccf681f75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:28.431947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aae015-6b7d-4248-9257-8dccf681f75b admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:28.432228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aae015-6b7d-4248-9257-8dccf681f75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:28.432502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aae015-6b7d-4248-9257-8dccf681f75b admin admin] PATH_INFO: `/v3/projects/4a62678586be437782ecf11c7aa2d6e9/users/63c8c6c414e14782a6f2fdc9863f0f49/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:28.447819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aae015-6b7d-4248-9257-8dccf681f75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a62678586be437782ecf11c7aa2d6e9, user_id=63c8c6c414e14782a6f2fdc9863f0f49, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:28.450463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aae015-6b7d-4248-9257-8dccf681f75b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:28.553359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1096/2214] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:28 2022] PUT /identity/v3/projects/4a62678586be437782ecf11c7aa2d6e9/users/63c8c6c414e14782a6f2fdc9863f0f49/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:28.557470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a49f-2fdc-4eb8-90fc-cb6be86aa986 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:28.571297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a49f-2fdc-4eb8-90fc-cb6be86aa986 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:28.635235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60a884-2f4e-457a-93bb-6afb9a405fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:28.635961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60a884-2f4e-457a-93bb-6afb9a405fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:28.636556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60a884-2f4e-457a-93bb-6afb9a405fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:28.636863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60a884-2f4e-457a-93bb-6afb9a405fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:28.637513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60a884-2f4e-457a-93bb-6afb9a405fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:28.639226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60a884-2f4e-457a-93bb-6afb9a405fbc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:28.652882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1119/2215] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:28.654088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aae015-6b7d-4248-9257-8dccf681f75b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:28.665362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aae015-6b7d-4248-9257-8dccf681f75b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:28.670439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:28.671115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:28.671318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:28.671676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] PATH_INFO: `/v3/projects/4a62678586be437782ecf11c7aa2d6e9/users/63c8c6c414e14782a6f2fdc9863f0f49/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:28.673732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a62678586be437782ecf11c7aa2d6e9, user_id=63c8c6c414e14782a6f2fdc9863f0f49, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:28.676502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:28.684367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:28.691448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1097/2216] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:28 2022] PUT /identity/v3/projects/4a62678586be437782ecf11c7aa2d6e9/users/63c8c6c414e14782a6f2fdc9863f0f49/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:28.702962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606373d-7468-4439-aeb8-08e350f9c9de None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:28.703291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606373d-7468-4439-aeb8-08e350f9c9de None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:28.703943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606373d-7468-4439-aeb8-08e350f9c9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:28.710971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:28.732520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5606373d-7468-4439-aeb8-08e350f9c9de None None] MFA Rules not processed for user `63c8c6c414e14782a6f2fdc9863f0f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:28.733766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5606373d-7468-4439-aeb8-08e350f9c9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:28.812396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1120/2217] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:28.837038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:28.837038 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:28.955277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945f38d-9da3-47c8-b770-f1cc03d5f256 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:28.959769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb06801-94b2-4cc5-8814-4eece4d524d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:28.960300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb06801-94b2-4cc5-8814-4eece4d524d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:28.960589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb06801-94b2-4cc5-8814-4eece4d524d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:28.960854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb06801-94b2-4cc5-8814-4eece4d524d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:28.971053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb06801-94b2-4cc5-8814-4eece4d524d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:28.973007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb06801-94b2-4cc5-8814-4eece4d524d3 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:28.985899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1098/2218] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:29.133177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606373d-7468-4439-aeb8-08e350f9c9de None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:29.134649 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5606373d-7468-4439-aeb8-08e350f9c9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:29.147758 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606373d-7468-4439-aeb8-08e350f9c9de None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:29.151343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6f50-f5b9-408d-8ab2-b644adaeaaa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:29.151950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6f50-f5b9-408d-8ab2-b644adaeaaa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:29.152355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6f50-f5b9-408d-8ab2-b644adaeaaa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:29.152491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6f50-f5b9-408d-8ab2-b644adaeaaa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:29.224541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e6f50-f5b9-408d-8ab2-b644adaeaaa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:29.226181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e6f50-f5b9-408d-8ab2-b644adaeaaa8 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:29.247939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1121/2219] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:30.410331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb06801-94b2-4cc5-8814-4eece4d524d3 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:30.430308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb06801-94b2-4cc5-8814-4eece4d524d3 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:30.436349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:30.437526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:30.438111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:30.438570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:30.443415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:30.513532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:30.515093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:30.525328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1099/2220] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:32.314748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6f50-f5b9-408d-8ab2-b644adaeaaa8 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:32.323895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6f50-f5b9-408d-8ab2-b644adaeaaa8 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:32.327692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea200f-230a-44e7-ab89-c6d622c83678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:32.328548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea200f-230a-44e7-ab89-c6d622c83678 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:32.328826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea200f-230a-44e7-ab89-c6d622c83678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:32.329149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea200f-230a-44e7-ab89-c6d622c83678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:32.330935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-faea200f-230a-44e7-ab89-c6d622c83678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:32.397395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea200f-230a-44e7-ab89-c6d622c83678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:32.398833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea200f-230a-44e7-ab89-c6d622c83678 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:32.408454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1122/2221] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:33.328478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:33.467199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b9203-d76c-4ca5-91e0-bc8748849924 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:33.478833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:33.479518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:33.479798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:33.480075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] PATH_INFO: `/v3/users/4a02fafc3b004b51959d344e4c1bb24f` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:33.481248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a02fafc3b004b51959d344e4c1bb24f)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:33.502855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:33.737087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:33.740690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a02fafc3b004b51959d344e4c1bb24f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:33.753457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a02fafc3b004b51959d344e4c1bb24f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:33.760497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a02fafc3b004b51959d344e4c1bb24f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:33.761218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a02fafc3b004b51959d344e4c1bb24f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:33.762138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a02fafc3b004b51959d344e4c1bb24f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:33.849884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1100/2222] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:33 2022] DELETE /identity/v3/users/4a02fafc3b004b51959d344e4c1bb24f => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:33.854617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32f77e-880a-4ccd-a1d8-101b409aa844 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:33.855184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32f77e-880a-4ccd-a1d8-101b409aa844 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:33.855278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32f77e-880a-4ccd-a1d8-101b409aa844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:33.874390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7e32f77e-880a-4ccd-a1d8-101b409aa844 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:33.875634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7e32f77e-880a-4ccd-a1d8-101b409aa844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:33.891278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:33.936823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1123/2223] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:33.949237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:33.950650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:34.013351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e359d59-ab21-4290-b615-c402f4f6595f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:34.017065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e156-f3b6-408d-91aa-2209071e5960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:34.017676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e156-f3b6-408d-91aa-2209071e5960 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:34.018091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e156-f3b6-408d-91aa-2209071e5960 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:34.018375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e156-f3b6-408d-91aa-2209071e5960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:34.029745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e156-f3b6-408d-91aa-2209071e5960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:34.031282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e156-f3b6-408d-91aa-2209071e5960 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:34.041398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1101/2224] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:23:34.573383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32f77e-880a-4ccd-a1d8-101b409aa844 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:34.575071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7e32f77e-880a-4ccd-a1d8-101b409aa844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:34.612662 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32f77e-880a-4ccd-a1d8-101b409aa844 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:34.618334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:34.619272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:34.619718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:34.619992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] PATH_INFO: `/v3/projects/8e6f37ea0d454f37b39b21f3dc73290a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:34.621206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6f37ea0d454f37b39b21f3dc73290a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:34.623297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:34.753986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6f37ea0d454f37b39b21f3dc73290a', 'request_id': 'req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:34.821418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6f37ea0d454f37b39b21f3dc73290a', 'request_id': 'req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:34.827426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6f37ea0d454f37b39b21f3dc73290a', 'request_id': 'req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:34.839285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] The token cache is being invalidate because project 8e6f37ea0d454f37b39b21f3dc73290a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:34.843494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:34.843494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:34.843494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1124/2225] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:34 2022] DELETE /identity/v3/projects/8e6f37ea0d454f37b39b21f3dc73290a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:34.845908 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e156-f3b6-408d-91aa-2209071e5960 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:34.856203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e156-f3b6-408d-91aa-2209071e5960 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:34.911068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:34.911687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:34.911964 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:34.912232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] PATH_INFO: `/v3/users/63c8c6c414e14782a6f2fdc9863f0f49` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:34.913343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c8c6c414e14782a6f2fdc9863f0f49)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:34.915269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:35.127607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c8c6c414e14782a6f2fdc9863f0f49'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:35.140498 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c8c6c414e14782a6f2fdc9863f0f49'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:35.142595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:35.149638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c8c6c414e14782a6f2fdc9863f0f49'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:35.150371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c8c6c414e14782a6f2fdc9863f0f49'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:35.150972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-5eeef5d1-16de-4e92-b51f-3c9073e109f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c8c6c414e14782a6f2fdc9863f0f49'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:35.227254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1102/2226] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:34 2022] DELETE /identity/v3/users/63c8c6c414e14782a6f2fdc9863f0f49 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:35.755580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:35.762488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:35.777847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe6aa7-3aab-4632-8f99-9bdebcec94fc admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:35.872928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:35.873588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:35.873870 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:35.874144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] PATH_INFO: `/v3/projects/4a62678586be437782ecf11c7aa2d6e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:35.875454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a62678586be437782ecf11c7aa2d6e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:35.880832 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:35.989160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:35.993503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a62678586be437782ecf11c7aa2d6e9', 'request_id': 'req-defb257a-a300-4969-adba-2385d0e14454'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:36.067725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a62678586be437782ecf11c7aa2d6e9', 'request_id': 'req-defb257a-a300-4969-adba-2385d0e14454'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:36.074213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a62678586be437782ecf11c7aa2d6e9', 'request_id': 'req-defb257a-a300-4969-adba-2385d0e14454'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:36.085974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] The token cache is being invalidate because project 4a62678586be437782ecf11c7aa2d6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:36.086310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:36.086930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-defb257a-a300-4969-adba-2385d0e14454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:36.088442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1125/2227] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:35 2022] DELETE /identity/v3/projects/4a62678586be437782ecf11c7aa2d6e9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:36.093411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aeb884-8083-4eb7-9b46-b5c74d183c33 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.095109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aeb884-8083-4eb7-9b46-b5c74d183c33 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.095109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aeb884-8083-4eb7-9b46-b5c74d183c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.113683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-85aeb884-8083-4eb7-9b46-b5c74d183c33 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:36.118082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-85aeb884-8083-4eb7-9b46-b5c74d183c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:36.174716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1103/2228] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:36.183367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb69a-c789-459f-9b57-3e19950d1315 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.183656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb69a-c789-459f-9b57-3e19950d1315 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.183932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb69a-c789-459f-9b57-3e19950d1315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.203954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cafeb69a-c789-459f-9b57-3e19950d1315 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:36.205141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cafeb69a-c789-459f-9b57-3e19950d1315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:36.213524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1126/2229] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:36.217939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aeb884-8083-4eb7-9b46-b5c74d183c33 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:36.228659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aeb884-8083-4eb7-9b46-b5c74d183c33 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:36.233343 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:36.233945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.234227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.234521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.235266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:23:36.235571 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:36.237184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:36.249457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1104/2230] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:23:36.253622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafeb69a-c789-459f-9b57-3e19950d1315 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:36.263551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafeb69a-c789-459f-9b57-3e19950d1315 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:36.268311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10539e9a-5ee2-4460-8a37-7199ad3b4e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:36.268906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10539e9a-5ee2-4460-8a37-7199ad3b4e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.269185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10539e9a-5ee2-4460-8a37-7199ad3b4e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.269459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10539e9a-5ee2-4460-8a37-7199ad3b4e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.270131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10539e9a-5ee2-4460-8a37-7199ad3b4e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:36.272245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10539e9a-5ee2-4460-8a37-7199ad3b4e9d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:36.353904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1127/2231] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:36 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:23:36.358057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:36.369299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c2ea9-213d-42e9-ab40-d54a11c3996a admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:36.444414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b5155-cb39-45e7-a48e-065034e6815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:36.444904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5155-cb39-45e7-a48e-065034e6815c admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.445181 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5155-cb39-45e7-a48e-065034e6815c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.445449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5155-cb39-45e7-a48e-065034e6815c admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.446066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b5155-cb39-45e7-a48e-065034e6815c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:36.448045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b5155-cb39-45e7-a48e-065034e6815c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:36.560666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1105/2232] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:36 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:36.564867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10539e9a-5ee2-4460-8a37-7199ad3b4e9d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:36.576136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10539e9a-5ee2-4460-8a37-7199ad3b4e9d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:36.580731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc147-d40a-4140-b48b-8ec4846d0366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:36.581332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc147-d40a-4140-b48b-8ec4846d0366 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.581612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc147-d40a-4140-b48b-8ec4846d0366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.581884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc147-d40a-4140-b48b-8ec4846d0366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.582529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc147-d40a-4140-b48b-8ec4846d0366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:36.584205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc147-d40a-4140-b48b-8ec4846d0366 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:36.594473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1128/2233] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:36.598415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b5155-cb39-45e7-a48e-065034e6815c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:36.609092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b5155-cb39-45e7-a48e-065034e6815c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:36.613842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1180709-ef6b-46f6-9631-e1f329b9c801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:36.615067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1180709-ef6b-46f6-9631-e1f329b9c801 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.615320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1180709-ef6b-46f6-9631-e1f329b9c801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.615587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1180709-ef6b-46f6-9631-e1f329b9c801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.616356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1180709-ef6b-46f6-9631-e1f329b9c801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:36.617858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1180709-ef6b-46f6-9631-e1f329b9c801 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:36.632305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1106/2234] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:36.636310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc147-d40a-4140-b48b-8ec4846d0366 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:36.647404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc147-d40a-4140-b48b-8ec4846d0366 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:36.652091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:36.652740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.653023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.653299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] PATH_INFO: `/v3/projects/76d541d88f464b5a8f23467069c08a2b/users/5ae3e7274a144e74839ba078833cae63/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.669403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d541d88f464b5a8f23467069c08a2b, user_id=5ae3e7274a144e74839ba078833cae63, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:36.672072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:36.764737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1129/2235] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:36 2022] PUT /identity/v3/projects/76d541d88f464b5a8f23467069c08a2b/users/5ae3e7274a144e74839ba078833cae63/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:36.776364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83c7b9-2e9d-4071-b094-ddc54fc2e21d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.776668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83c7b9-2e9d-4071-b094-ddc54fc2e21d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.776939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83c7b9-2e9d-4071-b094-ddc54fc2e21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.805845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-5a83c7b9-2e9d-4071-b094-ddc54fc2e21d None None] MFA Rules not processed for user `5ae3e7274a144e74839ba078833cae63`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:36.807091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5a83c7b9-2e9d-4071-b094-ddc54fc2e21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:36.844309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:36.869685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1107/2236] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:36.881183 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:36.884448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:36.943564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4d63-d425-4a36-84b5-89fb7c52da19 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:36.947264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50cc1d-8586-43d0-9d83-1c6fd6e0fa1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b5ee9e4388d4bbabc7f1e9e3ca50b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:36.947875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50cc1d-8586-43d0-9d83-1c6fd6e0fa1e service glance] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:36.948148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50cc1d-8586-43d0-9d83-1c6fd6e0fa1e service glance] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:36.948423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50cc1d-8586-43d0-9d83-1c6fd6e0fa1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:36.956813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50cc1d-8586-43d0-9d83-1c6fd6e0fa1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:36.958871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50cc1d-8586-43d0-9d83-1c6fd6e0fa1e service glance] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:36.968173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1130/2237] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:40.324217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83c7b9-2e9d-4071-b094-ddc54fc2e21d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:40.326010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5a83c7b9-2e9d-4071-b094-ddc54fc2e21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:40.403086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83c7b9-2e9d-4071-b094-ddc54fc2e21d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:40.408426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:40.409068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:40.409351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:40.409641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] PATH_INFO: `/v3/users/6fafd14760f84b3fa41ffd97929a91b4` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:40.414119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fafd14760f84b3fa41ffd97929a91b4)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:40.416262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:40.578975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fafd14760f84b3fa41ffd97929a91b4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:40.593149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fafd14760f84b3fa41ffd97929a91b4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:40.605259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fafd14760f84b3fa41ffd97929a91b4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:40.605259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fafd14760f84b3fa41ffd97929a91b4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:40.605480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fafd14760f84b3fa41ffd97929a91b4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:40.678589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1108/2238] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:40 2022] DELETE /identity/v3/users/6fafd14760f84b3fa41ffd97929a91b4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:40.680030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b49d2-b9e6-4aeb-b057-2c8055a1f43c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:40.680945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b49d2-b9e6-4aeb-b057-2c8055a1f43c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:40.681231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b49d2-b9e6-4aeb-b057-2c8055a1f43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:40.703689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3d3b49d2-b9e6-4aeb-b057-2c8055a1f43c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:40.704951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3d3b49d2-b9e6-4aeb-b057-2c8055a1f43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:40.781754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1131/2239] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:40.792290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:40.793781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:40.857166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f118ae0-6bc0-466c-9fec-04f28a83a0c9 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:40.860845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216988-5d5f-49a1-8fa6-832921482a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:40.861451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216988-5d5f-49a1-8fa6-832921482a81 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:40.861729 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216988-5d5f-49a1-8fa6-832921482a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:40.862014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216988-5d5f-49a1-8fa6-832921482a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:40.872312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23216988-5d5f-49a1-8fa6-832921482a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:40.873747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23216988-5d5f-49a1-8fa6-832921482a81 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:40.878808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:40.883553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1109/2240] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:41.435470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b49d2-b9e6-4aeb-b057-2c8055a1f43c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:41.436544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3d3b49d2-b9e6-4aeb-b057-2c8055a1f43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:41.448783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b49d2-b9e6-4aeb-b057-2c8055a1f43c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:41.454016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:41.461092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:41.461092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:41.461092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] PATH_INFO: `/v3/projects/2d0d271d466a4b40ae02a5a89b76b71f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:41.461092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d0d271d466a4b40ae02a5a89b76b71f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:41.461092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:41.485220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:41.591746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d0d271d466a4b40ae02a5a89b76b71f', 'request_id': 'req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:41.688839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d0d271d466a4b40ae02a5a89b76b71f', 'request_id': 'req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:41.696030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d0d271d466a4b40ae02a5a89b76b71f', 'request_id': 'req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:41.708830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] The token cache is being invalidate because project 2d0d271d466a4b40ae02a5a89b76b71f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:41.709135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:41.709811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:41.711365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1132/2241] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:41 2022] DELETE /identity/v3/projects/2d0d271d466a4b40ae02a5a89b76b71f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:41.715492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216988-5d5f-49a1-8fa6-832921482a81 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:41.726090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216988-5d5f-49a1-8fa6-832921482a81 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:41.788569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:41.789326 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:41.789609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:41.789910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] PATH_INFO: `/v3/users/bec50d15fbae42cba748b23dd3769a87` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:41.791057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec50d15fbae42cba748b23dd3769a87)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:41.793004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:42.017083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec50d15fbae42cba748b23dd3769a87'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.034598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec50d15fbae42cba748b23dd3769a87'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.046207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec50d15fbae42cba748b23dd3769a87'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.047193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec50d15fbae42cba748b23dd3769a87'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.048091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec50d15fbae42cba748b23dd3769a87'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.123732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1110/2242] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:41 2022] DELETE /identity/v3/users/bec50d15fbae42cba748b23dd3769a87 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:42.555944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:42.557639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:42.576704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb782f-59a6-4ca4-bcda-a4e36cf1a62d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:42.649665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:42.650326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:42.650609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:42.650923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] PATH_INFO: `/v3/projects/1d7a9ec0e541495bb9f15650b3b1b8b1` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:42.652124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7a9ec0e541495bb9f15650b3b1b8b1)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:42.654389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:42.781825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7a9ec0e541495bb9f15650b3b1b8b1', 'request_id': 'req-cd41f0cc-8fda-41c0-b2c9-372556db97e1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.849539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7a9ec0e541495bb9f15650b3b1b8b1', 'request_id': 'req-cd41f0cc-8fda-41c0-b2c9-372556db97e1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.856137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7a9ec0e541495bb9f15650b3b1b8b1', 'request_id': 'req-cd41f0cc-8fda-41c0-b2c9-372556db97e1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.868169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] The token cache is being invalidate because project 1d7a9ec0e541495bb9f15650b3b1b8b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:42.868450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.869267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:42.871353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1133/2243] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:42 2022] DELETE /identity/v3/projects/1d7a9ec0e541495bb9f15650b3b1b8b1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:42.875224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:42.876697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:42.890258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff9936-d9d1-4802-b0d8-7cfda7cc4069 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:42.953493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:42.954124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:42.954399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:42.954669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] PATH_INFO: `/v3/users/2f129cfede2a47859fd76b9deda1e1df` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:42.955826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f129cfede2a47859fd76b9deda1e1df)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:42.959504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:42.991910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:43.205994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f129cfede2a47859fd76b9deda1e1df'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:43.220734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f129cfede2a47859fd76b9deda1e1df'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:43.229130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f129cfede2a47859fd76b9deda1e1df'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:43.229858 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f129cfede2a47859fd76b9deda1e1df'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:43.230457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f129cfede2a47859fd76b9deda1e1df'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:43.340731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1111/2244] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:42 2022] DELETE /identity/v3/users/2f129cfede2a47859fd76b9deda1e1df => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:43.858780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:43.860289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:43.873885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41f0cc-8fda-41c0-b2c9-372556db97e1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:43.900652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:43.934696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:43.935373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:43.935650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:43.935948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] PATH_INFO: `/v3/projects/88abd9f8a61e4d77a78e86bd8b4e38c2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:43.937102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88abd9f8a61e4d77a78e86bd8b4e38c2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:43.939057 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:44.082306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88abd9f8a61e4d77a78e86bd8b4e38c2', 'request_id': 'req-487a0ea5-db9c-44bf-92f0-d99f6364def3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.105803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:44.107977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:44.121927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2223ca7-9303-4261-9b10-378962e9d6f8 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:44.178360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88abd9f8a61e4d77a78e86bd8b4e38c2', 'request_id': 'req-487a0ea5-db9c-44bf-92f0-d99f6364def3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.191330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88abd9f8a61e4d77a78e86bd8b4e38c2', 'request_id': 'req-487a0ea5-db9c-44bf-92f0-d99f6364def3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.197657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] The token cache is being invalidate because project 88abd9f8a61e4d77a78e86bd8b4e38c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:44.198031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.198927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-487a0ea5-db9c-44bf-92f0-d99f6364def3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.200742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1134/2245] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:43 2022] DELETE /identity/v3/projects/88abd9f8a61e4d77a78e86bd8b4e38c2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:44.205818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83040edc-3783-4ebc-9e80-4934159959a4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.206766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83040edc-3783-4ebc-9e80-4934159959a4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.206766 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83040edc-3783-4ebc-9e80-4934159959a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.226548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:44.227331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.227617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.227897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] PATH_INFO: `/v3/users/02394fc2ef2448d8a7e390ab13598c91` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.229841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02394fc2ef2448d8a7e390ab13598c91)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:44.232084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:44.236207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-83040edc-3783-4ebc-9e80-4934159959a4 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:44.237826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-83040edc-3783-4ebc-9e80-4934159959a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:44.249891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1135/2246] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:44.267223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346ffa-3646-41ef-8eba-5bbae3d7974a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.267627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346ffa-3646-41ef-8eba-5bbae3d7974a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.267907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346ffa-3646-41ef-8eba-5bbae3d7974a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.299325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-38346ffa-3646-41ef-8eba-5bbae3d7974a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:44.303697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-38346ffa-3646-41ef-8eba-5bbae3d7974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:44.312109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1136/2247] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:44.316405 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346ffa-3646-41ef-8eba-5bbae3d7974a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:44.326062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346ffa-3646-41ef-8eba-5bbae3d7974a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:44.331039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:44.331642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.331920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.332201 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.332962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:23:44.333742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:44.335404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:44.352389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1137/2248] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:23:44.353470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:44.371357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18268b12-4311-417a-9d52-437dd4453606 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:44.376920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:44.377570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.378790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.379111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.380651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:44.389527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:44.461735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02394fc2ef2448d8a7e390ab13598c91'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.475597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02394fc2ef2448d8a7e390ab13598c91'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.483114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02394fc2ef2448d8a7e390ab13598c91'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.483901 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02394fc2ef2448d8a7e390ab13598c91'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.484616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2dd98f7f-9398-4ce6-ad39-cf30376c2b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02394fc2ef2448d8a7e390ab13598c91'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:44.502310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1138/2249] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:44 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:44.506811 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:44.508517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:44.561292 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1112/2250] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:44 2022] DELETE /identity/v3/users/02394fc2ef2448d8a7e390ab13598c91 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:44.566128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98573a-970a-4d29-8f60-a3afedcc712d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.566420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98573a-970a-4d29-8f60-a3afedcc712d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.566784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98573a-970a-4d29-8f60-a3afedcc712d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.608594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc24d6d-8860-4701-a21b-739565ff58ea admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:44.611771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4d98573a-970a-4d29-8f60-a3afedcc712d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:44.616226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4d98573a-970a-4d29-8f60-a3afedcc712d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:44.628779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:44.638288 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:44.704421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365f91b-bb53-4283-a076-0800c5129d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:44.705075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365f91b-bb53-4283-a076-0800c5129d1c admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.705341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365f91b-bb53-4283-a076-0800c5129d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.705620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365f91b-bb53-4283-a076-0800c5129d1c admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.706874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365f91b-bb53-4283-a076-0800c5129d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:44.720685 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365f91b-bb53-4283-a076-0800c5129d1c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:44.726050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1113/2251] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:23:44.742869 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98573a-970a-4d29-8f60-a3afedcc712d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:44.746693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4d98573a-970a-4d29-8f60-a3afedcc712d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:44.813211 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1139/2252] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:44 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:23:44.814194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365f91b-bb53-4283-a076-0800c5129d1c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:44.818423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98573a-970a-4d29-8f60-a3afedcc712d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:44.823268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bd0f0-e783-4d39-a724-ff5d27378ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:44.824093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bd0f0-e783-4d39-a724-ff5d27378ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.824689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bd0f0-e783-4d39-a724-ff5d27378ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.824974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bd0f0-e783-4d39-a724-ff5d27378ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.831129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365f91b-bb53-4283-a076-0800c5129d1c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:44.840120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bd0f0-e783-4d39-a724-ff5d27378ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:44.843662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bd0f0-e783-4d39-a724-ff5d27378ac7 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:44.847707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a6c3c-7da9-4e43-893f-9544591e6f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:44.849410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a6c3c-7da9-4e43-893f-9544591e6f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.849956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a6c3c-7da9-4e43-893f-9544591e6f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.850722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a6c3c-7da9-4e43-893f-9544591e6f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.851888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a6c3c-7da9-4e43-893f-9544591e6f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:44.854397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a6c3c-7da9-4e43-893f-9544591e6f1e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:44.854624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1114/2253] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:44.867822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1140/2254] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:44.871981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bd0f0-e783-4d39-a724-ff5d27378ac7 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:44.887677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bd0f0-e783-4d39-a724-ff5d27378ac7 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:44.892737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08161b02-df26-488c-b93e-e2ae34de68a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:44.893531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08161b02-df26-488c-b93e-e2ae34de68a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.893886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08161b02-df26-488c-b93e-e2ae34de68a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.894620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08161b02-df26-488c-b93e-e2ae34de68a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.894975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08161b02-df26-488c-b93e-e2ae34de68a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:44.897184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08161b02-df26-488c-b93e-e2ae34de68a6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:44.915195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1115/2255] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:44.915381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a6c3c-7da9-4e43-893f-9544591e6f1e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:44.928442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a6c3c-7da9-4e43-893f-9544591e6f1e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:44.933798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:44.934865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:44.935200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:44.935479 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] PATH_INFO: `/v3/projects/74e6b27298e34383acb124fd4fc56918/users/522765afd64c4018baebae36facfc3f3/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:44.955974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e6b27298e34383acb124fd4fc56918, user_id=522765afd64c4018baebae36facfc3f3, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:44.958752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:45.040422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1141/2256] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:44 2022] PUT /identity/v3/projects/74e6b27298e34383acb124fd4fc56918/users/522765afd64c4018baebae36facfc3f3/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:23:45.045433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291cb7a-a206-45e7-90d5-033f5bd9ba61 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:45.045724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291cb7a-a206-45e7-90d5-033f5bd9ba61 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:45.046666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291cb7a-a206-45e7-90d5-033f5bd9ba61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:45.068601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2291cb7a-a206-45e7-90d5-033f5bd9ba61 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:45.070626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2291cb7a-a206-45e7-90d5-033f5bd9ba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:45.148098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1116/2257] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:45.160504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:45.162755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:45.289352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ed5f9-199c-40cb-98e0-0275018cde25 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:45.300717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7861fe0-cc64-47dd-80c7-7d1b0bb542ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:45.301415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7861fe0-cc64-47dd-80c7-7d1b0bb542ef service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:45.301697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7861fe0-cc64-47dd-80c7-7d1b0bb542ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:45.302719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7861fe0-cc64-47dd-80c7-7d1b0bb542ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:45.316280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7861fe0-cc64-47dd-80c7-7d1b0bb542ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:45.318190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7861fe0-cc64-47dd-80c7-7d1b0bb542ef service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:45.330930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1142/2258] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:45.545643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291cb7a-a206-45e7-90d5-033f5bd9ba61 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:45.548096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2291cb7a-a206-45e7-90d5-033f5bd9ba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:45.567602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291cb7a-a206-45e7-90d5-033f5bd9ba61 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:45.576223 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:45.586861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:45.587212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:45.587480 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] PATH_INFO: `/v3/projects/701c6a7ff3e244cf93b3f7756f6b56a9` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:45.589560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=701c6a7ff3e244cf93b3f7756f6b56a9)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:45.591546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:45.763752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:45.766676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701c6a7ff3e244cf93b3f7756f6b56a9', 'request_id': 'req-b97d4075-f033-43e8-977c-3e9550c016e6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:45.856683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701c6a7ff3e244cf93b3f7756f6b56a9', 'request_id': 'req-b97d4075-f033-43e8-977c-3e9550c016e6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:45.863088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701c6a7ff3e244cf93b3f7756f6b56a9', 'request_id': 'req-b97d4075-f033-43e8-977c-3e9550c016e6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:45.876882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] The token cache is being invalidate because project 701c6a7ff3e244cf93b3f7756f6b56a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:45.877174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:45.877940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-b97d4075-f033-43e8-977c-3e9550c016e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:45.879482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1117/2259] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:45 2022] DELETE /identity/v3/projects/701c6a7ff3e244cf93b3f7756f6b56a9 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:45.885241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3aab2-7d2a-45d6-a24e-83d01c97b459 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:45.887068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3aab2-7d2a-45d6-a24e-83d01c97b459 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:45.887513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3aab2-7d2a-45d6-a24e-83d01c97b459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:45.910032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-77a3aab2-7d2a-45d6-a24e-83d01c97b459 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:45.911639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-77a3aab2-7d2a-45d6-a24e-83d01c97b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:45.997560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1143/2260] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:23:46.023990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd4faf-7319-4320-9422-719ec9bd5d16 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:46.024309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd4faf-7319-4320-9422-719ec9bd5d16 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:46.024547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd4faf-7319-4320-9422-719ec9bd5d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:46.044742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-43fd4faf-7319-4320-9422-719ec9bd5d16 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:46.045784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-43fd4faf-7319-4320-9422-719ec9bd5d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:46.054702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1118/2261] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:46.059125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3aab2-7d2a-45d6-a24e-83d01c97b459 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:46.070425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3aab2-7d2a-45d6-a24e-83d01c97b459 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:46.077603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:46.078224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:46.078540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:46.079484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:46.080195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:23:46.080498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:46.082192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:46.093061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:46.096551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1144/2262] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:23:46.101535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd4faf-7319-4320-9422-719ec9bd5d16 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:46.112582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd4faf-7319-4320-9422-719ec9bd5d16 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:46.121894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a9dc7-b807-4b69-b5c6-5c7e2ccfcbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:46.122602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a9dc7-b807-4b69-b5c6-5c7e2ccfcbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:46.122963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a9dc7-b807-4b69-b5c6-5c7e2ccfcbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:46.123327 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a9dc7-b807-4b69-b5c6-5c7e2ccfcbd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:46.124059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a9dc7-b807-4b69-b5c6-5c7e2ccfcbd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:46.126169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a9dc7-b807-4b69-b5c6-5c7e2ccfcbd9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:46.371265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1119/2263] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:46 2022] POST /identity/v3/projects => generated 385 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:23:46.375968 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:46.390044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb2faf-1f5a-490e-b5bf-d1aa73cab1a6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:46.457836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446b455-315b-4dc4-9feb-050fe210bfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:46.458623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446b455-315b-4dc4-9feb-050fe210bfa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:46.458917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446b455-315b-4dc4-9feb-050fe210bfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:46.459221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446b455-315b-4dc4-9feb-050fe210bfa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:46.459883 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446b455-315b-4dc4-9feb-050fe210bfa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:46.461906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446b455-315b-4dc4-9feb-050fe210bfa2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:46.562849 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1145/2264] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:46 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:23:46.567030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a9dc7-b807-4b69-b5c6-5c7e2ccfcbd9 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:46.583056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a9dc7-b807-4b69-b5c6-5c7e2ccfcbd9 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:46.599449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99a828-1cae-42d1-b952-fdebf276da1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:46.599449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99a828-1cae-42d1-b952-fdebf276da1a admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:46.599449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99a828-1cae-42d1-b952-fdebf276da1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:46.599449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99a828-1cae-42d1-b952-fdebf276da1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:46.599783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99a828-1cae-42d1-b952-fdebf276da1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:46.599783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99a828-1cae-42d1-b952-fdebf276da1a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:46.605435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1120/2265] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:46.610666 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446b455-315b-4dc4-9feb-050fe210bfa2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:46.622705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446b455-315b-4dc4-9feb-050fe210bfa2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:46.630445 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da678f-d0c1-4bb3-941d-19f0fb7c64ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:46.631205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da678f-d0c1-4bb3-941d-19f0fb7c64ce admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:46.631653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da678f-d0c1-4bb3-941d-19f0fb7c64ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:46.632181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da678f-d0c1-4bb3-941d-19f0fb7c64ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:46.633325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da678f-d0c1-4bb3-941d-19f0fb7c64ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:46.635424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da678f-d0c1-4bb3-941d-19f0fb7c64ce admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:46.647558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1146/2266] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:46.652263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99a828-1cae-42d1-b952-fdebf276da1a admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:46.662575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99a828-1cae-42d1-b952-fdebf276da1a admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:46.667872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:46.668707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:46.669077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:46.669262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] PATH_INFO: `/v3/projects/da3dc1fd07c642ac926b7a298b453d8d/users/1c8a04101924401394738855291a36fc/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:46.692002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3dc1fd07c642ac926b7a298b453d8d, user_id=1c8a04101924401394738855291a36fc, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:46.694987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:46.823966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1121/2267] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:46 2022] PUT /identity/v3/projects/da3dc1fd07c642ac926b7a298b453d8d/users/1c8a04101924401394738855291a36fc/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:46.827280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a854f-77de-4f27-ba26-0d7fddfbec5a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:46.827858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a854f-77de-4f27-ba26-0d7fddfbec5a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:46.828412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a854f-77de-4f27-ba26-0d7fddfbec5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:46.867848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-529a854f-77de-4f27-ba26-0d7fddfbec5a None None] MFA Rules not processed for user `1c8a04101924401394738855291a36fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:46.869087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-529a854f-77de-4f27-ba26-0d7fddfbec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:46.936563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1147/2268] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:46.948359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:46.950115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:47.017494 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45041e0-feb7-4d80-8d7e-94a334414aea admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:47.022083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:47.022617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:47.030594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:47.030594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:47.032086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:47.033658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:47.045843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1122/2269] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:47.093858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892236d-6235-4a0b-b96f-530fde03021f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:47.094154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892236d-6235-4a0b-b96f-530fde03021f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:47.094427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892236d-6235-4a0b-b96f-530fde03021f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:47.140159 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b892236d-6235-4a0b-b96f-530fde03021f None None] MFA Rules not processed for user `522765afd64c4018baebae36facfc3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:47.142252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b892236d-6235-4a0b-b96f-530fde03021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:47.232046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:47.238853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1148/2270] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:47.250053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:47.251615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:47.263882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80f8f-a3a9-4b51-9270-0a19d0d9b844 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:47.268002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ee365-b8ba-4d18-b6a2-070fa3e3bcdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:47.268829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ee365-b8ba-4d18-b6a2-070fa3e3bcdd service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:47.269113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ee365-b8ba-4d18-b6a2-070fa3e3bcdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:47.269392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ee365-b8ba-4d18-b6a2-070fa3e3bcdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:47.282342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ee365-b8ba-4d18-b6a2-070fa3e3bcdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:47.284147 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ee365-b8ba-4d18-b6a2-070fa3e3bcdd service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:47.294055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1123/2271] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:50.123623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cbcaa-e2c8-43ef-b6b1-bcee1c969696 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:50.124172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cbcaa-e2c8-43ef-b6b1-bcee1c969696 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:50.124172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cbcaa-e2c8-43ef-b6b1-bcee1c969696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:50.143807 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7a3cbcaa-e2c8-43ef-b6b1-bcee1c969696 None None] MFA Rules not processed for user `522765afd64c4018baebae36facfc3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:50.145055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7a3cbcaa-e2c8-43ef-b6b1-bcee1c969696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:50.153077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1149/2272] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:50.372209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ee365-b8ba-4d18-b6a2-070fa3e3bcdd service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:50.439261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ee365-b8ba-4d18-b6a2-070fa3e3bcdd service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:50.444267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:50.444923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:50.445202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:50.445489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] PATH_INFO: `/v3/users/1c8a04101924401394738855291a36fc` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:50.446652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8a04101924401394738855291a36fc)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:50.448618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:50.450159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:50.669719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8a04101924401394738855291a36fc'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:50.681835 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8a04101924401394738855291a36fc'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:50.688566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8a04101924401394738855291a36fc'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:50.689243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8a04101924401394738855291a36fc'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:50.689924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8a04101924401394738855291a36fc'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:50.769336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1124/2273] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:23:50 2022] DELETE /identity/v3/users/1c8a04101924401394738855291a36fc => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:50.773867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2990a9-1164-4149-a1aa-258b2c9fc3db None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:50.774160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2990a9-1164-4149-a1aa-258b2c9fc3db None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:50.774457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2990a9-1164-4149-a1aa-258b2c9fc3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:50.793293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4e2990a9-1164-4149-a1aa-258b2c9fc3db None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:50.794530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4e2990a9-1164-4149-a1aa-258b2c9fc3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:50.854548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1150/2274] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:50.864744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:50.866143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:50.932159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad2ce9-30bd-4a99-bb58-e7eddd1cfbff admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:50.936296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b034571-98da-4174-b7eb-765d2006bdf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:50.936911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b034571-98da-4174-b7eb-765d2006bdf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:50.937189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b034571-98da-4174-b7eb-765d2006bdf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:50.937461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b034571-98da-4174-b7eb-765d2006bdf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:50.947977 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b034571-98da-4174-b7eb-765d2006bdf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:50.949329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b034571-98da-4174-b7eb-765d2006bdf7 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:50.959401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1125/2275] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:23:51.394989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2990a9-1164-4149-a1aa-258b2c9fc3db None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:51.397391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4e2990a9-1164-4149-a1aa-258b2c9fc3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:51.409948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2990a9-1164-4149-a1aa-258b2c9fc3db None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:51.424271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:51.425038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:51.425421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:51.425764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] PATH_INFO: `/v3/projects/da3dc1fd07c642ac926b7a298b453d8d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:51.427187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3dc1fd07c642ac926b7a298b453d8d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:51.429200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:51.532917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3dc1fd07c642ac926b7a298b453d8d', 'request_id': 'req-0b81240f-45cd-4860-83fe-b429f8c4c19c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:51.592004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3dc1fd07c642ac926b7a298b453d8d', 'request_id': 'req-0b81240f-45cd-4860-83fe-b429f8c4c19c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:51.598505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3dc1fd07c642ac926b7a298b453d8d', 'request_id': 'req-0b81240f-45cd-4860-83fe-b429f8c4c19c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:51.610268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] The token cache is being invalidate because project da3dc1fd07c642ac926b7a298b453d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:23:51.610679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:51.611404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0b81240f-45cd-4860-83fe-b429f8c4c19c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:23:51.613510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1151/2276] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:23:51 2022] DELETE /identity/v3/projects/da3dc1fd07c642ac926b7a298b453d8d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:51.618967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33cf2-dd68-4b56-9e1c-3a8b9a588ffb None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:51.619510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33cf2-dd68-4b56-9e1c-3a8b9a588ffb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:51.619598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33cf2-dd68-4b56-9e1c-3a8b9a588ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:51.643221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4cf33cf2-dd68-4b56-9e1c-3a8b9a588ffb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:51.646910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4cf33cf2-dd68-4b56-9e1c-3a8b9a588ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:51.709448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1126/2277] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:51.717256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669de0d-0d9d-4e24-9333-c87d0015ae67 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:51.717940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669de0d-0d9d-4e24-9333-c87d0015ae67 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:51.717940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669de0d-0d9d-4e24-9333-c87d0015ae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:51.738025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7669de0d-0d9d-4e24-9333-c87d0015ae67 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:51.739809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7669de0d-0d9d-4e24-9333-c87d0015ae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:51.748069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1152/2278] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:51.752470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf33cf2-dd68-4b56-9e1c-3a8b9a588ffb None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:51.763981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf33cf2-dd68-4b56-9e1c-3a8b9a588ffb None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:51.768683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:51.773154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:51.773154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:51.773154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:51.773154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:23:51.773154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:51.773154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:51.785975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1127/2279] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:23:51.790498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669de0d-0d9d-4e24-9333-c87d0015ae67 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:51.802661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669de0d-0d9d-4e24-9333-c87d0015ae67 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:51.807736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c86a7-34d5-4ce1-8151-e57536e2464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:51.808375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c86a7-34d5-4ce1-8151-e57536e2464a admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:51.808648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c86a7-34d5-4ce1-8151-e57536e2464a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:51.808954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c86a7-34d5-4ce1-8151-e57536e2464a admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:51.809601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c86a7-34d5-4ce1-8151-e57536e2464a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:51.811580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c86a7-34d5-4ce1-8151-e57536e2464a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:51.914988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1153/2280] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:23:51 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:23:51.916258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:51.926525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f890b-eba9-451a-92f2-9d946c05210c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:51.932626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:52.007219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e61b6-abec-4cd9-8474-1409ab92ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:52.008089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e61b6-abec-4cd9-8474-1409ab92ab4e admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:52.008551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e61b6-abec-4cd9-8474-1409ab92ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:52.008985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e61b6-abec-4cd9-8474-1409ab92ab4e admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:52.009911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e61b6-abec-4cd9-8474-1409ab92ab4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:52.012957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e61b6-abec-4cd9-8474-1409ab92ab4e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:52.119142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1128/2281] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:23:51 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:23:52.123363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c86a7-34d5-4ce1-8151-e57536e2464a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:52.138614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c86a7-34d5-4ce1-8151-e57536e2464a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:52.141320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e2260-a6e2-46e7-99cd-fc91d81dc8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:52.141922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e2260-a6e2-46e7-99cd-fc91d81dc8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:52.142207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e2260-a6e2-46e7-99cd-fc91d81dc8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:52.142479 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e2260-a6e2-46e7-99cd-fc91d81dc8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:52.143151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e2260-a6e2-46e7-99cd-fc91d81dc8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:52.144736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e2260-a6e2-46e7-99cd-fc91d81dc8cf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:52.155146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1154/2282] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:52.159078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e61b6-abec-4cd9-8474-1409ab92ab4e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:52.169819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e61b6-abec-4cd9-8474-1409ab92ab4e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:52.174404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38267c-6a18-44e4-aa3a-57be9fdb36c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:52.175018 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38267c-6a18-44e4-aa3a-57be9fdb36c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:52.175311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38267c-6a18-44e4-aa3a-57be9fdb36c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:52.175586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38267c-6a18-44e4-aa3a-57be9fdb36c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:52.176227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38267c-6a18-44e4-aa3a-57be9fdb36c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:52.178394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38267c-6a18-44e4-aa3a-57be9fdb36c3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:52.189074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1129/2283] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:23:52.193311 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e2260-a6e2-46e7-99cd-fc91d81dc8cf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:52.203334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e2260-a6e2-46e7-99cd-fc91d81dc8cf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:52.208670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:52.209339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:52.209582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:52.209857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] PATH_INFO: `/v3/projects/ea2d7ec47b41495ba87b76a5773ebbe7/users/f5158c442683481baf95c7973e97655f/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:52.227205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea2d7ec47b41495ba87b76a5773ebbe7, user_id=f5158c442683481baf95c7973e97655f, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:52.229846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:52.375463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1155/2284] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:23:52 2022] PUT /identity/v3/projects/ea2d7ec47b41495ba87b76a5773ebbe7/users/f5158c442683481baf95c7973e97655f/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:23:52.385769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9911d9-315d-4939-87e9-b5794b839a0c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:52.386052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9911d9-315d-4939-87e9-b5794b839a0c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:52.386332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9911d9-315d-4939-87e9-b5794b839a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:52.416819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-ed9911d9-315d-4939-87e9-b5794b839a0c None None] MFA Rules not processed for user `f5158c442683481baf95c7973e97655f`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:23:52.418044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ed9911d9-315d-4939-87e9-b5794b839a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:52.482558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1130/2285] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:23:52.493349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:23:52.495092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:23:52.509120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:23:52.555528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349db43-2f1a-4aa3-9932-4c951c9a9c4a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:23:52.561836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec50bfa-384c-4247-9ca7-e626522ee3eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:23:52.562638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50bfa-384c-4247-9ca7-e626522ee3eb service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:23:52.562638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50bfa-384c-4247-9ca7-e626522ee3eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:23:52.563343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50bfa-384c-4247-9ca7-e626522ee3eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:23:52.572636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec50bfa-384c-4247-9ca7-e626522ee3eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:23:52.573673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec50bfa-384c-4247-9ca7-e626522ee3eb service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:23:52.584230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1156/2286] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:23:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:02.479752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:02.598729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=21 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:03.875305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9911d9-315d-4939-87e9-b5794b839a0c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:03.938524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9911d9-315d-4939-87e9-b5794b839a0c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:03.942946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:03.943644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:03.943933 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:03.944209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] PATH_INFO: `/v3/users/f5158c442683481baf95c7973e97655f` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:03.945656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5158c442683481baf95c7973e97655f)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:03.947648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:04.187291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5158c442683481baf95c7973e97655f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:04.201984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5158c442683481baf95c7973e97655f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:04.210166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5158c442683481baf95c7973e97655f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:04.211461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5158c442683481baf95c7973e97655f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:04.212091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5158c442683481baf95c7973e97655f'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:04.291636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1131/2287] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:24:03 2022] DELETE /identity/v3/users/f5158c442683481baf95c7973e97655f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:04.295951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de9c03-494e-4200-ae8b-74cff90b5a4a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:04.296456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de9c03-494e-4200-ae8b-74cff90b5a4a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:04.296644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de9c03-494e-4200-ae8b-74cff90b5a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:04.315928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-98de9c03-494e-4200-ae8b-74cff90b5a4a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:04.317285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-98de9c03-494e-4200-ae8b-74cff90b5a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:04.381274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1157/2288] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:04.391602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:04.393116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:04.452894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85978c10-04cb-46b5-8181-19fbd21d68a8 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:04.456402 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfe911-adfd-48c5-b74d-ebcd1635880d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:04.457006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfe911-adfd-48c5-b74d-ebcd1635880d service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:04.457282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfe911-adfd-48c5-b74d-ebcd1635880d service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:04.457555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfe911-adfd-48c5-b74d-ebcd1635880d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:04.466886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfe911-adfd-48c5-b74d-ebcd1635880d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:04.468356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfe911-adfd-48c5-b74d-ebcd1635880d service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:04.478485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1132/2289] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:04.984417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de9c03-494e-4200-ae8b-74cff90b5a4a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:04.985845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-98de9c03-494e-4200-ae8b-74cff90b5a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:04.998395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de9c03-494e-4200-ae8b-74cff90b5a4a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:05.003964 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:05.004753 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.005001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.005279 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] PATH_INFO: `/v3/projects/ea2d7ec47b41495ba87b76a5773ebbe7` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.006595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea2d7ec47b41495ba87b76a5773ebbe7)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:05.008810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:05.104521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea2d7ec47b41495ba87b76a5773ebbe7', 'request_id': 'req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:05.165980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea2d7ec47b41495ba87b76a5773ebbe7', 'request_id': 'req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:05.172241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea2d7ec47b41495ba87b76a5773ebbe7', 'request_id': 'req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:05.183725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] The token cache is being invalidate because project ea2d7ec47b41495ba87b76a5773ebbe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:24:05.184018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:05.184650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08143cb4-c4bf-44e7-807b-b4a414d4e6d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:05.186126 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1158/2290] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:24:04 2022] DELETE /identity/v3/projects/ea2d7ec47b41495ba87b76a5773ebbe7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:05.191424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efff891-a4cb-4ef2-9f34-5f6562cbd14f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.191911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efff891-a4cb-4ef2-9f34-5f6562cbd14f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.192004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efff891-a4cb-4ef2-9f34-5f6562cbd14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.213666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1efff891-a4cb-4ef2-9f34-5f6562cbd14f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:05.214872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1efff891-a4cb-4ef2-9f34-5f6562cbd14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:05.233131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:05.281261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1133/2291] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:05.287475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0734f62-21c4-4acf-a521-3d943007fa3a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.287841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0734f62-21c4-4acf-a521-3d943007fa3a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.288215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0734f62-21c4-4acf-a521-3d943007fa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.312485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b0734f62-21c4-4acf-a521-3d943007fa3a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:05.313700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b0734f62-21c4-4acf-a521-3d943007fa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:05.324891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1159/2292] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:24:05.329153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efff891-a4cb-4ef2-9f34-5f6562cbd14f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:05.339834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efff891-a4cb-4ef2-9f34-5f6562cbd14f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:05.346278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:05.347098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.347450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.347855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.348544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:24:05.348956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:05.350772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:05.364768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1134/2293] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:24:05.369632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0734f62-21c4-4acf-a521-3d943007fa3a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:05.385488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0734f62-21c4-4acf-a521-3d943007fa3a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:05.392210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff9798-01a9-40c4-ac41-603c358d1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:05.397861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff9798-01a9-40c4-ac41-603c358d1f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.397861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff9798-01a9-40c4-ac41-603c358d1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.397861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff9798-01a9-40c4-ac41-603c358d1f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.397861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff9798-01a9-40c4-ac41-603c358d1f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:05.397861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff9798-01a9-40c4-ac41-603c358d1f04 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:05.534724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1160/2294] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:24:05 2022] POST /identity/v3/projects => generated 397 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:24:05.539943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:05.551823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe8f14-0c38-411f-9f52-59e97216bb48 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:05.628423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28fed-c99e-4573-8b7f-7cb499654ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:05.629190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28fed-c99e-4573-8b7f-7cb499654ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.629473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28fed-c99e-4573-8b7f-7cb499654ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.629748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28fed-c99e-4573-8b7f-7cb499654ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.630423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b28fed-c99e-4573-8b7f-7cb499654ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:05.632409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b28fed-c99e-4573-8b7f-7cb499654ad2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:05.724382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1135/2295] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:24:05 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:24:05.729818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff9798-01a9-40c4-ac41-603c358d1f04 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:05.744713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff9798-01a9-40c4-ac41-603c358d1f04 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:05.767432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b92c4-6018-4bfd-a561-a21f58e08a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:05.768146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b92c4-6018-4bfd-a561-a21f58e08a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.772437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b92c4-6018-4bfd-a561-a21f58e08a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.772437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b92c4-6018-4bfd-a561-a21f58e08a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.772437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b92c4-6018-4bfd-a561-a21f58e08a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:05.772437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b92c4-6018-4bfd-a561-a21f58e08a97 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:05.784187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1161/2296] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:05.788219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28fed-c99e-4573-8b7f-7cb499654ad2 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:05.800579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28fed-c99e-4573-8b7f-7cb499654ad2 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:05.807349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af2a81-f012-42af-a337-048598668314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:05.808224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2a81-f012-42af-a337-048598668314 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.808576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2a81-f012-42af-a337-048598668314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.808914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2a81-f012-42af-a337-048598668314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.809750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af2a81-f012-42af-a337-048598668314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:05.811760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af2a81-f012-42af-a337-048598668314 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:05.824800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1136/2297] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:05.833837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b92c4-6018-4bfd-a561-a21f58e08a97 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:05.845932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b92c4-6018-4bfd-a561-a21f58e08a97 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:05.852917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:05.853582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.853861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.854150 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] PATH_INFO: `/v3/projects/7b96e5ed734942fdb5803c7583857720/users/8129677e7936477ca16185814a29a99c/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.874727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b96e5ed734942fdb5803c7583857720, user_id=8129677e7936477ca16185814a29a99c, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:05.877530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:05.946114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1162/2298] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:24:05 2022] PUT /identity/v3/projects/7b96e5ed734942fdb5803c7583857720/users/8129677e7936477ca16185814a29a99c/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:05.956232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce04754-8028-4028-90e2-a36379d1844d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:05.956586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce04754-8028-4028-90e2-a36379d1844d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:05.956935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce04754-8028-4028-90e2-a36379d1844d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:05.985372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3ce04754-8028-4028-90e2-a36379d1844d None None] MFA Rules not processed for user `8129677e7936477ca16185814a29a99c`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:05.986590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3ce04754-8028-4028-90e2-a36379d1844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:06.052396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1137/2299] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:06.063791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:06.065494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:06.089156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:06.130575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd90be-6d74-4e65-8c73-4eef0d5d193c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:06.135109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:06.135720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:06.135999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:06.136269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:06.144765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:06.146989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:06.156569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1163/2300] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:07.384875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce04754-8028-4028-90e2-a36379d1844d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:07.386318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3ce04754-8028-4028-90e2-a36379d1844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:07.406461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:07.464182 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce04754-8028-4028-90e2-a36379d1844d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:07.467859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74659866-7b14-407e-8778-d8fd9830f840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:07.468470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659866-7b14-407e-8778-d8fd9830f840 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:07.468747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659866-7b14-407e-8778-d8fd9830f840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:07.469016 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659866-7b14-407e-8778-d8fd9830f840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:07.470750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-74659866-7b14-407e-8778-d8fd9830f840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:07.560473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74659866-7b14-407e-8778-d8fd9830f840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:07.562000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74659866-7b14-407e-8778-d8fd9830f840 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:07.574863 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1138/2301] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:09.775834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:09.777394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:09.844152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61a83c-848b-4408-a91e-b6eafe7c2e6c service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:09.852127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:09.852749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:09.852944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:09.853225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] PATH_INFO: `/v3/users/5ae3e7274a144e74839ba078833cae63` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:09.854374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae3e7274a144e74839ba078833cae63)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:09.857837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:10.086636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae3e7274a144e74839ba078833cae63'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:10.094301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae3e7274a144e74839ba078833cae63'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:10.111783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae3e7274a144e74839ba078833cae63'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:10.112627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae3e7274a144e74839ba078833cae63'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:10.113314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae3e7274a144e74839ba078833cae63'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:10.183744 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1164/2302] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:24:09 2022] DELETE /identity/v3/users/5ae3e7274a144e74839ba078833cae63 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:10.188095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cbd78-7664-457c-a946-1c20e0dab1be None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:10.188574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cbd78-7664-457c-a946-1c20e0dab1be None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:10.188671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cbd78-7664-457c-a946-1c20e0dab1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:10.208245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-761cbd78-7664-457c-a946-1c20e0dab1be None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:10.209459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-761cbd78-7664-457c-a946-1c20e0dab1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:10.271998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1139/2303] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:10.281910 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:10.283353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:10.330409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:10.342534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89374dfd-9e19-4b2f-9bb6-bffbe27d2802 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:10.346226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5b1ab-4083-4758-ad47-0aef250cda93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:10.346828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b1ab-4083-4758-ad47-0aef250cda93 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:10.347148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b1ab-4083-4758-ad47-0aef250cda93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:10.347424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b1ab-4083-4758-ad47-0aef250cda93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:10.356816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5b1ab-4083-4758-ad47-0aef250cda93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:10.358262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5b1ab-4083-4758-ad47-0aef250cda93 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:10.368870 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1165/2304] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:10.832674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cbd78-7664-457c-a946-1c20e0dab1be None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:10.834556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-761cbd78-7664-457c-a946-1c20e0dab1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:10.848652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cbd78-7664-457c-a946-1c20e0dab1be None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:10.855383 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:10.855991 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:10.856276 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:10.856553 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] PATH_INFO: `/v3/projects/76d541d88f464b5a8f23467069c08a2b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:10.857726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d541d88f464b5a8f23467069c08a2b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:10.859860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:10.983275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d541d88f464b5a8f23467069c08a2b', 'request_id': 'req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:11.073859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d541d88f464b5a8f23467069c08a2b', 'request_id': 'req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:11.079177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:11.080549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d541d88f464b5a8f23467069c08a2b', 'request_id': 'req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:11.096540 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] The token cache is being invalidate because project 76d541d88f464b5a8f23467069c08a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:24:11.097124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:11.097894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-44af5dd8-0d3f-4d44-9f23-7f6039afbaf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:11.108957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1140/2305] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:24:10 2022] DELETE /identity/v3/projects/76d541d88f464b5a8f23467069c08a2b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:11.118253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924a90-6cbb-449e-aa65-de9cad0b0e3f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.118736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924a90-6cbb-449e-aa65-de9cad0b0e3f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.118817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924a90-6cbb-449e-aa65-de9cad0b0e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.145817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5a924a90-6cbb-449e-aa65-de9cad0b0e3f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:11.147390 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5a924a90-6cbb-449e-aa65-de9cad0b0e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:11.208358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1166/2306] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:11.217240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6645eb-6557-468f-aba6-9cf31644d481 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.217775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6645eb-6557-468f-aba6-9cf31644d481 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.217775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6645eb-6557-468f-aba6-9cf31644d481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.238886 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bf6645eb-6557-468f-aba6-9cf31644d481 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:11.240178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bf6645eb-6557-468f-aba6-9cf31644d481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:11.248144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1141/2307] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:11.252331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a924a90-6cbb-449e-aa65-de9cad0b0e3f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:11.263281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a924a90-6cbb-449e-aa65-de9cad0b0e3f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:11.267115 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:11.267804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.268154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.268503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.269259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:24:11.269634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:11.271369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:11.283695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1167/2308] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:24:11.299252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6645eb-6557-468f-aba6-9cf31644d481 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:11.308662 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6645eb-6557-468f-aba6-9cf31644d481 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:11.313436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef696c-3c9f-4377-983f-5352260e752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:11.314088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef696c-3c9f-4377-983f-5352260e752a admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.314367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef696c-3c9f-4377-983f-5352260e752a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.314642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef696c-3c9f-4377-983f-5352260e752a admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.315345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef696c-3c9f-4377-983f-5352260e752a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:11.317221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef696c-3c9f-4377-983f-5352260e752a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:11.394622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1142/2309] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:24:11 2022] POST /identity/v3/projects => generated 361 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:24:11.399585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:11.409689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1c1d5-6474-4226-aee4-c1c934face13 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:11.466514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e36e97-98e7-4b1f-a1ac-a610c6bafa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:11.467143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e36e97-98e7-4b1f-a1ac-a610c6bafa14 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.467424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e36e97-98e7-4b1f-a1ac-a610c6bafa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.467694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e36e97-98e7-4b1f-a1ac-a610c6bafa14 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.468313 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e36e97-98e7-4b1f-a1ac-a610c6bafa14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:11.470219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e36e97-98e7-4b1f-a1ac-a610c6bafa14 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:11.536399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1168/2310] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:24:11 2022] POST /identity/v3/users => generated 281 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:24:11.540520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef696c-3c9f-4377-983f-5352260e752a admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:11.550770 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef696c-3c9f-4377-983f-5352260e752a admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:11.555631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228c66f-f67d-4fd9-bed2-47a63369f072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:11.556255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c66f-f67d-4fd9-bed2-47a63369f072 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.556564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c66f-f67d-4fd9-bed2-47a63369f072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.556841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c66f-f67d-4fd9-bed2-47a63369f072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.557515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228c66f-f67d-4fd9-bed2-47a63369f072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:11.559149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228c66f-f67d-4fd9-bed2-47a63369f072 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:11.569571 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1143/2311] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:11.573843 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e36e97-98e7-4b1f-a1ac-a610c6bafa14 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:11.583829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e36e97-98e7-4b1f-a1ac-a610c6bafa14 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:11.589238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c8f93-68f3-45e0-9596-0ceb5ee15164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:11.589831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8f93-68f3-45e0-9596-0ceb5ee15164 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.590111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8f93-68f3-45e0-9596-0ceb5ee15164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.590383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8f93-68f3-45e0-9596-0ceb5ee15164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.591053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c8f93-68f3-45e0-9596-0ceb5ee15164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:11.592627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c8f93-68f3-45e0-9596-0ceb5ee15164 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:11.603714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1169/2312] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:11.607765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228c66f-f67d-4fd9-bed2-47a63369f072 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:11.618203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228c66f-f67d-4fd9-bed2-47a63369f072 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:11.622822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:11.623487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.623767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.624040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] PATH_INFO: `/v3/projects/1273a9869f02483f852489043442f746/users/b638e6f05b2e45f7965953224409d2c0/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.638810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1273a9869f02483f852489043442f746, user_id=b638e6f05b2e45f7965953224409d2c0, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:11.641457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:11.711635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1144/2313] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:24:11 2022] PUT /identity/v3/projects/1273a9869f02483f852489043442f746/users/b638e6f05b2e45f7965953224409d2c0/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:11.722423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb556ab0-62dc-472d-b838-ac5a5bca81b8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.722707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb556ab0-62dc-472d-b838-ac5a5bca81b8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.723130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb556ab0-62dc-472d-b838-ac5a5bca81b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.752230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fb556ab0-62dc-472d-b838-ac5a5bca81b8 None None] MFA Rules not processed for user `b638e6f05b2e45f7965953224409d2c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:11.753431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fb556ab0-62dc-472d-b838-ac5a5bca81b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:11.796771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:11.825602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1170/2314] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:11.861667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:11.864041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:11.924411 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cee92-da61-4639-8375-8c33249e0d31 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:11.927883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33abc8-1d79-410d-9755-7452f6cc39d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:11.928507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33abc8-1d79-410d-9755-7452f6cc39d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:11.928808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33abc8-1d79-410d-9755-7452f6cc39d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:11.929074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33abc8-1d79-410d-9755-7452f6cc39d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:11.940148 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33abc8-1d79-410d-9755-7452f6cc39d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:11.941004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33abc8-1d79-410d-9755-7452f6cc39d2 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:11.950068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1145/2315] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:11.965418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb556ab0-62dc-472d-b838-ac5a5bca81b8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:12.030412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb556ab0-62dc-472d-b838-ac5a5bca81b8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:12.034853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:12.035486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:12.035763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:12.036030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] PATH_INFO: `/v3/users/b638e6f05b2e45f7965953224409d2c0` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:12.037207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b638e6f05b2e45f7965953224409d2c0)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:12.039137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:12.249615 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b638e6f05b2e45f7965953224409d2c0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:12.256606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b638e6f05b2e45f7965953224409d2c0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:12.269055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b638e6f05b2e45f7965953224409d2c0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:12.270748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b638e6f05b2e45f7965953224409d2c0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:12.270748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b638e6f05b2e45f7965953224409d2c0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:12.365675 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1171/2316] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:24:11 2022] DELETE /identity/v3/users/b638e6f05b2e45f7965953224409d2c0 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:12.370052 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e002059-7e44-49f6-b0cf-8daa1c60d032 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:12.370554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e002059-7e44-49f6-b0cf-8daa1c60d032 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:12.370736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e002059-7e44-49f6-b0cf-8daa1c60d032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:12.391349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-9e002059-7e44-49f6-b0cf-8daa1c60d032 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:12.392572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9e002059-7e44-49f6-b0cf-8daa1c60d032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:12.452680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1146/2317] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:24:12.462441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:12.463923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:12.521173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c299d-a3f9-465a-a66c-5aed9b20189a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:12.524951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f0197-d337-4166-97c0-762b7fe6702b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:12.525552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f0197-d337-4166-97c0-762b7fe6702b service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:12.525834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f0197-d337-4166-97c0-762b7fe6702b service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:12.526110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f0197-d337-4166-97c0-762b7fe6702b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:12.534976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f0197-d337-4166-97c0-762b7fe6702b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:12.536459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f0197-d337-4166-97c0-762b7fe6702b service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:12.545956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1172/2318] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:12.950297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e002059-7e44-49f6-b0cf-8daa1c60d032 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:12.952232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9e002059-7e44-49f6-b0cf-8daa1c60d032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:12.974551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e002059-7e44-49f6-b0cf-8daa1c60d032 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:12.983697 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:12.985606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:12.986089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:12.987801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] PATH_INFO: `/v3/projects/1273a9869f02483f852489043442f746` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.005856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1273a9869f02483f852489043442f746)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:13.005856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:13.132845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:13.152671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1273a9869f02483f852489043442f746', 'request_id': 'req-ea6e1afe-91c5-4802-bcd3-d0cafb051184'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:13.221822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1273a9869f02483f852489043442f746', 'request_id': 'req-ea6e1afe-91c5-4802-bcd3-d0cafb051184'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:13.228707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1273a9869f02483f852489043442f746', 'request_id': 'req-ea6e1afe-91c5-4802-bcd3-d0cafb051184'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:13.241220 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] The token cache is being invalidate because project 1273a9869f02483f852489043442f746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:24:13.241635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:13.242299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ea6e1afe-91c5-4802-bcd3-d0cafb051184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:13.244013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1147/2319] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:24:12 2022] DELETE /identity/v3/projects/1273a9869f02483f852489043442f746 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:13.249587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f9a9d-11ac-4733-b283-1b481aa10a13 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:13.249895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f9a9d-11ac-4733-b283-1b481aa10a13 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:13.250428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f9a9d-11ac-4733-b283-1b481aa10a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.258046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:13.272184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-575f9a9d-11ac-4733-b283-1b481aa10a13 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:13.273459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-575f9a9d-11ac-4733-b283-1b481aa10a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:13.345061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1173/2320] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:13.354773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e608fc6-0d12-4cc0-a15a-37e4182839d6 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:13.355091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e608fc6-0d12-4cc0-a15a-37e4182839d6 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:13.355364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e608fc6-0d12-4cc0-a15a-37e4182839d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.375383 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6e608fc6-0d12-4cc0-a15a-37e4182839d6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:13.376575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6e608fc6-0d12-4cc0-a15a-37e4182839d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:13.385138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1148/2321] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:13.389667 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f9a9d-11ac-4733-b283-1b481aa10a13 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:13.399582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f9a9d-11ac-4733-b283-1b481aa10a13 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:13.405424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:13.405966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:13.406250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:13.406528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.407260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:24:13.407567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:13.409237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:13.421792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1174/2322] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:24:13.426155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e608fc6-0d12-4cc0-a15a-37e4182839d6 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:13.437646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e608fc6-0d12-4cc0-a15a-37e4182839d6 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:13.442300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d14564-9618-4d77-954d-bcf193a50046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:13.442890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d14564-9618-4d77-954d-bcf193a50046 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:13.443253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d14564-9618-4d77-954d-bcf193a50046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:13.443511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d14564-9618-4d77-954d-bcf193a50046 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.444135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d14564-9618-4d77-954d-bcf193a50046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:13.446023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d14564-9618-4d77-954d-bcf193a50046 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:13.613090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1149/2323] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:24:13 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:24:13.617167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:13.627103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876da42-3253-47f7-b4de-11c0d7989e0c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:13.683466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a7cd7-1ec6-4ea5-a6b6-0fdde4728b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:13.683988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a7cd7-1ec6-4ea5-a6b6-0fdde4728b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:13.684269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a7cd7-1ec6-4ea5-a6b6-0fdde4728b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:13.684542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a7cd7-1ec6-4ea5-a6b6-0fdde4728b81 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.685152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a7cd7-1ec6-4ea5-a6b6-0fdde4728b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:13.687070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a7cd7-1ec6-4ea5-a6b6-0fdde4728b81 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:13.827779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1175/2324] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:24:13 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:24:13.829282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d14564-9618-4d77-954d-bcf193a50046 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:13.841670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d14564-9618-4d77-954d-bcf193a50046 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:13.847694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c32e5-b848-4bee-b734-b24e8d49d436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:13.848336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c32e5-b848-4bee-b734-b24e8d49d436 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:13.848676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c32e5-b848-4bee-b734-b24e8d49d436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:13.848947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c32e5-b848-4bee-b734-b24e8d49d436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.849602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c32e5-b848-4bee-b734-b24e8d49d436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:13.851444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c32e5-b848-4bee-b734-b24e8d49d436 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:13.866163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1150/2325] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:13.867724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a7cd7-1ec6-4ea5-a6b6-0fdde4728b81 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:13.882679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a7cd7-1ec6-4ea5-a6b6-0fdde4728b81 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:13.888510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4173d-2a3f-41fd-965a-c1dd984a9ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:13.889152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4173d-2a3f-41fd-965a-c1dd984a9ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:13.889455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4173d-2a3f-41fd-965a-c1dd984a9ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:13.889727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4173d-2a3f-41fd-965a-c1dd984a9ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.890374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4173d-2a3f-41fd-965a-c1dd984a9ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:13.892243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4173d-2a3f-41fd-965a-c1dd984a9ec9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:13.904185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1176/2326] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:24:13.914725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c32e5-b848-4bee-b734-b24e8d49d436 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:13.925702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c32e5-b848-4bee-b734-b24e8d49d436 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:13.930419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:13.931123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:13.931403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:13.931674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] PATH_INFO: `/v3/projects/b01305a41c0c4fb0a8ac6f95bbe02f24/users/583440def64a469590349c93b501f536/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:13.950296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01305a41c0c4fb0a8ac6f95bbe02f24, user_id=583440def64a469590349c93b501f536, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:13.951882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:14.035671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1151/2327] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:24:13 2022] PUT /identity/v3/projects/b01305a41c0c4fb0a8ac6f95bbe02f24/users/583440def64a469590349c93b501f536/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:14.045836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f5994-4388-4b73-9f31-c4a6486f09b0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:14.046255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f5994-4388-4b73-9f31-c4a6486f09b0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:14.046657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f5994-4388-4b73-9f31-c4a6486f09b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:14.082580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-aa3f5994-4388-4b73-9f31-c4a6486f09b0 None None] MFA Rules not processed for user `583440def64a469590349c93b501f536`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:14.085001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-aa3f5994-4388-4b73-9f31-c4a6486f09b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:14.155999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:14.163153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1177/2328] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:14.174734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:14.176911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:14.250884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6f637-6ab4-4423-914d-ed1c6656c9e5 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:14.256660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0dbc1c-15d0-4383-97ef-dbc7d70621e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:14.257445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dbc1c-15d0-4383-97ef-dbc7d70621e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:14.258269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dbc1c-15d0-4383-97ef-dbc7d70621e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:14.258618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dbc1c-15d0-4383-97ef-dbc7d70621e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:14.272816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0dbc1c-15d0-4383-97ef-dbc7d70621e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:14.274302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0dbc1c-15d0-4383-97ef-dbc7d70621e8 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:14.285355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1152/2329] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:16.202417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f5994-4388-4b73-9f31-c4a6486f09b0 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:16.204046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-aa3f5994-4388-4b73-9f31-c4a6486f09b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:16.283321 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f5994-4388-4b73-9f31-c4a6486f09b0 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:16.288161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:16.288794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:16.289085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:16.289356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] PATH_INFO: `/v3/users/8129677e7936477ca16185814a29a99c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:16.290509 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8129677e7936477ca16185814a29a99c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:16.292548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:16.508724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8129677e7936477ca16185814a29a99c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:16.518122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8129677e7936477ca16185814a29a99c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:16.531767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8129677e7936477ca16185814a29a99c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:16.532500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8129677e7936477ca16185814a29a99c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:16.533101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8129677e7936477ca16185814a29a99c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:16.671486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1178/2330] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:24:16 2022] DELETE /identity/v3/users/8129677e7936477ca16185814a29a99c => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:16.706546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0fa07-3dbc-42ab-937b-08beaf6998d8 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:16.706546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0fa07-3dbc-42ab-937b-08beaf6998d8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:16.706546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0fa07-3dbc-42ab-937b-08beaf6998d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:16.722717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e1f0fa07-3dbc-42ab-937b-08beaf6998d8 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:16.724101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e1f0fa07-3dbc-42ab-937b-08beaf6998d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:16.774746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:16.787986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1153/2331] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:16.797651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:16.799103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:16.858181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fbe92-1203-4e48-9e5e-f8ad4b9c30af admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:16.861730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36019581-ced2-4293-9e1e-6956b159ab9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:16.862334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36019581-ced2-4293-9e1e-6956b159ab9a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:16.862607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36019581-ced2-4293-9e1e-6956b159ab9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:16.862878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36019581-ced2-4293-9e1e-6956b159ab9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:16.873906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36019581-ced2-4293-9e1e-6956b159ab9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:16.875414 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36019581-ced2-4293-9e1e-6956b159ab9a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:16.884905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1179/2332] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:17.448726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0fa07-3dbc-42ab-937b-08beaf6998d8 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:17.448726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e1f0fa07-3dbc-42ab-937b-08beaf6998d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:17.487326 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0fa07-3dbc-42ab-937b-08beaf6998d8 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:17.489657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:17.490392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:17.490674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:17.490971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] PATH_INFO: `/v3/projects/7b96e5ed734942fdb5803c7583857720` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:17.492453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b96e5ed734942fdb5803c7583857720)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:17.494272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:17.670295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b96e5ed734942fdb5803c7583857720', 'request_id': 'req-4a5e781c-f34c-4963-89ff-3dd24972c728'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:17.759291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b96e5ed734942fdb5803c7583857720', 'request_id': 'req-4a5e781c-f34c-4963-89ff-3dd24972c728'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:17.766743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b96e5ed734942fdb5803c7583857720', 'request_id': 'req-4a5e781c-f34c-4963-89ff-3dd24972c728'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:17.801410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] The token cache is being invalidate because project 7b96e5ed734942fdb5803c7583857720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:24:17.802346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:17.803629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4a5e781c-f34c-4963-89ff-3dd24972c728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:17.806838 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1154/2333] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:24:17 2022] DELETE /identity/v3/projects/7b96e5ed734942fdb5803c7583857720 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:17.811834 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f507cd-b0a9-4053-a5ca-ec158525b843 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:17.813529 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f507cd-b0a9-4053-a5ca-ec158525b843 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:17.813981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f507cd-b0a9-4053-a5ca-ec158525b843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:17.836058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-74f507cd-b0a9-4053-a5ca-ec158525b843 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:17.837493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-74f507cd-b0a9-4053-a5ca-ec158525b843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:17.852297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:17.917022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1180/2334] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:17.924902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a1b8-e6f9-4949-aeb3-0e78c64868cd None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:17.925256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a1b8-e6f9-4949-aeb3-0e78c64868cd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:17.925602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a1b8-e6f9-4949-aeb3-0e78c64868cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:17.949672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2fe6a1b8-e6f9-4949-aeb3-0e78c64868cd None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:17.950795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2fe6a1b8-e6f9-4949-aeb3-0e78c64868cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:17.958907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1155/2335] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:17.963276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f507cd-b0a9-4053-a5ca-ec158525b843 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:17.973788 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f507cd-b0a9-4053-a5ca-ec158525b843 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:17.978676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:17.979314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:17.984952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:17.984952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:17.984952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:24:17.984952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:17.984952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:17.994251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1181/2336] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:24:17.998784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6a1b8-e6f9-4949-aeb3-0e78c64868cd None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:18.009915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6a1b8-e6f9-4949-aeb3-0e78c64868cd None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:18.014610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3a106-1998-49c1-b37a-80d685cad044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:18.015278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a106-1998-49c1-b37a-80d685cad044 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:18.015556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a106-1998-49c1-b37a-80d685cad044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:18.015831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a106-1998-49c1-b37a-80d685cad044 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:18.016501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3a106-1998-49c1-b37a-80d685cad044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:18.018417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3a106-1998-49c1-b37a-80d685cad044 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:18.104904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1156/2337] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:24:17 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:24:18.113724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:18.124212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e775279-5309-41d0-9e41-3c366838876e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:18.187235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb5057-82b4-4f14-9858-d30cb5fdf988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:18.190477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb5057-82b4-4f14-9858-d30cb5fdf988 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:18.190477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb5057-82b4-4f14-9858-d30cb5fdf988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:18.190477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb5057-82b4-4f14-9858-d30cb5fdf988 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:18.190477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb5057-82b4-4f14-9858-d30cb5fdf988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:18.192219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb5057-82b4-4f14-9858-d30cb5fdf988 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:18.325781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1182/2338] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:24:18 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:24:18.330259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3a106-1998-49c1-b37a-80d685cad044 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:18.342401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3a106-1998-49c1-b37a-80d685cad044 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:18.351367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3465c39-cb4b-4f50-aa71-c8219146f65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:18.351367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3465c39-cb4b-4f50-aa71-c8219146f65f admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:18.351367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3465c39-cb4b-4f50-aa71-c8219146f65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:18.351367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3465c39-cb4b-4f50-aa71-c8219146f65f admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:18.352159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3465c39-cb4b-4f50-aa71-c8219146f65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:18.352159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3465c39-cb4b-4f50-aa71-c8219146f65f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:18.366536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1157/2339] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:18.369722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb5057-82b4-4f14-9858-d30cb5fdf988 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:18.380551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb5057-82b4-4f14-9858-d30cb5fdf988 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:18.385172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f0033-37de-4b14-a73b-8b37cf41c6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:18.385768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f0033-37de-4b14-a73b-8b37cf41c6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:18.386053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f0033-37de-4b14-a73b-8b37cf41c6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:18.386384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f0033-37de-4b14-a73b-8b37cf41c6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:18.387410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f0033-37de-4b14-a73b-8b37cf41c6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:18.389255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f0033-37de-4b14-a73b-8b37cf41c6a2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:18.399875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1183/2340] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:18.404469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3465c39-cb4b-4f50-aa71-c8219146f65f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:18.414302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3465c39-cb4b-4f50-aa71-c8219146f65f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:18.421436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:18.421436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:18.421436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:18.421436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] PATH_INFO: `/v3/projects/006bf6eb97a14d8c8c33f880b0bb80bc/users/c62c3bfffc214c57a1f48b2779a615ac/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:18.437169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=006bf6eb97a14d8c8c33f880b0bb80bc, user_id=c62c3bfffc214c57a1f48b2779a615ac, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:18.439875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:18.543532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1158/2341] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:24:18 2022] PUT /identity/v3/projects/006bf6eb97a14d8c8c33f880b0bb80bc/users/c62c3bfffc214c57a1f48b2779a615ac/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:18.548637 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7dced-513b-410c-85e6-324286dc1e0d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:18.548928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7dced-513b-410c-85e6-324286dc1e0d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:18.549202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7dced-513b-410c-85e6-324286dc1e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:18.567718 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-98e7dced-513b-410c-85e6-324286dc1e0d None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:18.569296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-98e7dced-513b-410c-85e6-324286dc1e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:18.632759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1184/2342] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:18.642733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:18.644712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:18.704041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3c7b0-70e2-4ba2-b299-302f87c977d8 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:18.707680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62b60f-d491-4741-95da-a5e021a2cfcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:18.708278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62b60f-d491-4741-95da-a5e021a2cfcc service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:18.710438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62b60f-d491-4741-95da-a5e021a2cfcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:18.710438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62b60f-d491-4741-95da-a5e021a2cfcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:18.718686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62b60f-d491-4741-95da-a5e021a2cfcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:18.720356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62b60f-d491-4741-95da-a5e021a2cfcc service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:18.730174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1159/2343] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:19.275948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7dced-513b-410c-85e6-324286dc1e0d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:19.278013 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-98e7dced-513b-410c-85e6-324286dc1e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:19.290435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7dced-513b-410c-85e6-324286dc1e0d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:19.295566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:19.296307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:19.296672 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:19.297017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] PATH_INFO: `/v3/users/583440def64a469590349c93b501f536` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:19.298350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=583440def64a469590349c93b501f536)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:19.300495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:19.332826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:19.514205 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583440def64a469590349c93b501f536'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:19.524778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583440def64a469590349c93b501f536'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:19.553280 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583440def64a469590349c93b501f536'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:19.554101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583440def64a469590349c93b501f536'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:19.555118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583440def64a469590349c93b501f536'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:19.649052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1185/2344] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:24:19 2022] DELETE /identity/v3/users/583440def64a469590349c93b501f536 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:24:19.654592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467892a-a08d-4f00-87b3-254d50bdba21 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:19.655221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467892a-a08d-4f00-87b3-254d50bdba21 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:19.655317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467892a-a08d-4f00-87b3-254d50bdba21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:19.677140 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8467892a-a08d-4f00-87b3-254d50bdba21 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:19.678667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8467892a-a08d-4f00-87b3-254d50bdba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:19.689575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:19.741358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1160/2345] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:19.751238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:19.752800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:19.829899 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32a5ee-ea68-4d5a-9c9a-0ceb1b627a6f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:19.838711 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13ae67-4ffb-423c-a9bd-dda3b75e4249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:19.839342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13ae67-4ffb-423c-a9bd-dda3b75e4249 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:19.840207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13ae67-4ffb-423c-a9bd-dda3b75e4249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:19.840469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13ae67-4ffb-423c-a9bd-dda3b75e4249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:19.864486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13ae67-4ffb-423c-a9bd-dda3b75e4249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:19.875190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13ae67-4ffb-423c-a9bd-dda3b75e4249 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:19.888185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1186/2346] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:20.403361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467892a-a08d-4f00-87b3-254d50bdba21 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:20.404190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8467892a-a08d-4f00-87b3-254d50bdba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:20.429516 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467892a-a08d-4f00-87b3-254d50bdba21 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:20.438891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:20.439705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:20.440095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:20.441591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] PATH_INFO: `/v3/projects/b01305a41c0c4fb0a8ac6f95bbe02f24` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:20.442912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01305a41c0c4fb0a8ac6f95bbe02f24)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:20.446044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:20.604461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01305a41c0c4fb0a8ac6f95bbe02f24', 'request_id': 'req-7dcb579d-01d0-4916-99f1-121725ef04ac'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:20.607954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a136c1f-65e6-4a34-8fa2-52c7011f8bd5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:20.608768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a136c1f-65e6-4a34-8fa2-52c7011f8bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:20.609200 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a136c1f-65e6-4a34-8fa2-52c7011f8bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:20.629264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5a136c1f-65e6-4a34-8fa2-52c7011f8bd5 None None] MFA Rules not processed for user `90dc7b751dd14c6f80205e709826423b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:20.630495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5a136c1f-65e6-4a34-8fa2-52c7011f8bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:20.674347 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01305a41c0c4fb0a8ac6f95bbe02f24', 'request_id': 'req-7dcb579d-01d0-4916-99f1-121725ef04ac'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:20.693842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01305a41c0c4fb0a8ac6f95bbe02f24', 'request_id': 'req-7dcb579d-01d0-4916-99f1-121725ef04ac'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:20.708957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] The token cache is being invalidate because project b01305a41c0c4fb0a8ac6f95bbe02f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:24:20.709246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:20.711622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:24:20.712131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1187/2347] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:20.716446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1161/2348] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:24:20 2022] DELETE /identity/v3/projects/b01305a41c0c4fb0a8ac6f95bbe02f24 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:24.460858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d03584-6a2b-466d-b322-21eb133c1f1b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:24.460858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d03584-6a2b-466d-b322-21eb133c1f1b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:24.460858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d03584-6a2b-466d-b322-21eb133c1f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:24.473671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c9d03584-6a2b-466d-b322-21eb133c1f1b None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:24.474905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c9d03584-6a2b-466d-b322-21eb133c1f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:24.537003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:24.575127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1188/2349] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:24.634923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:24.637422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:24.727803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb579d-01d0-4916-99f1-121725ef04ac admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:24.734423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:24.735075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:24.735356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:24.735629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:24.745344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:24.748117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:24.757890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1162/2350] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:24:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:24:25.333645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9e42-5aa9-4c6f-a749-562069bdb3f5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:25.333926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9e42-5aa9-4c6f-a749-562069bdb3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:25.334405 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9e42-5aa9-4c6f-a749-562069bdb3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:25.392927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-afee9e42-5aa9-4c6f-a749-562069bdb3f5 None None] MFA Rules not processed for user `c62c3bfffc214c57a1f48b2779a615ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:25.394402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-afee9e42-5aa9-4c6f-a749-562069bdb3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:25.571385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1189/2351] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:25.572178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:25.574700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:25.696955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:25.781299 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d1516-9c63-4aab-8b46-70d42d7263df service swift] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:25.789249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4a61b-d7d8-46df-b53e-2767b49581b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:25.791536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a61b-d7d8-46df-b53e-2767b49581b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:25.792272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a61b-d7d8-46df-b53e-2767b49581b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:25.793601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a61b-d7d8-46df-b53e-2767b49581b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:25.795083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4a61b-d7d8-46df-b53e-2767b49581b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:25.799328 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4a61b-d7d8-46df-b53e-2767b49581b2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:25.948935 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1163/2352] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:24:25 2022] POST /identity/v3/projects => generated 383 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:24:25.953026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee9e42-5aa9-4c6f-a749-562069bdb3f5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:25.980971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee9e42-5aa9-4c6f-a749-562069bdb3f5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:26.097261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da763d82-249e-4e7e-add2-22dd171265b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:26.097877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da763d82-249e-4e7e-add2-22dd171265b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:26.098158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da763d82-249e-4e7e-add2-22dd171265b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:26.098427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da763d82-249e-4e7e-add2-22dd171265b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:26.099781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da763d82-249e-4e7e-add2-22dd171265b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:26.108337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da763d82-249e-4e7e-add2-22dd171265b2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:26.229067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4a61b-d7d8-46df-b53e-2767b49581b2 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:26.241049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1190/2353] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:24:25 2022] POST /identity/v3/users => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:24:26.244643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4a61b-d7d8-46df-b53e-2767b49581b2 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:26.250350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3ba48-fdad-4bc1-9597-649a2ec9cfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:26.250945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3ba48-fdad-4bc1-9597-649a2ec9cfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:26.251322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3ba48-fdad-4bc1-9597-649a2ec9cfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:26.251598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3ba48-fdad-4bc1-9597-649a2ec9cfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:26.252300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3ba48-fdad-4bc1-9597-649a2ec9cfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:26.253968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3ba48-fdad-4bc1-9597-649a2ec9cfe2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:26.268431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1164/2354] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:26.272494 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da763d82-249e-4e7e-add2-22dd171265b2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:26.283923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da763d82-249e-4e7e-add2-22dd171265b2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:26.288650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff148b58-0ae7-42a7-a994-896c92682561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:26.289396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff148b58-0ae7-42a7-a994-896c92682561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:26.289618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff148b58-0ae7-42a7-a994-896c92682561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:26.289985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff148b58-0ae7-42a7-a994-896c92682561 admin admin] PATH_INFO: `/v3/projects/3c53b0ab05544dec860358ffb05225b4/users/026a6d256c494b74988b2b200e936a11/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:26.312330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff148b58-0ae7-42a7-a994-896c92682561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c53b0ab05544dec860358ffb05225b4, user_id=026a6d256c494b74988b2b200e936a11, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:26.317740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff148b58-0ae7-42a7-a994-896c92682561 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:26.487613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1191/2355] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:24:26 2022] PUT /identity/v3/projects/3c53b0ab05544dec860358ffb05225b4/users/026a6d256c494b74988b2b200e936a11/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:24:26.487730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3ba48-fdad-4bc1-9597-649a2ec9cfe2 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:26.500322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3ba48-fdad-4bc1-9597-649a2ec9cfe2 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:26.581788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90a614-24da-4798-af1f-47e32b007417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:26.582528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a614-24da-4798-af1f-47e32b007417 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:26.583023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a614-24da-4798-af1f-47e32b007417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:26.583384 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a614-24da-4798-af1f-47e32b007417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:26.584289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90a614-24da-4798-af1f-47e32b007417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:26.586356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90a614-24da-4798-af1f-47e32b007417 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:26.602367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff148b58-0ae7-42a7-a994-896c92682561 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:26.603222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1165/2356] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:24:26.613781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff148b58-0ae7-42a7-a994-896c92682561 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:26.618995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01080c5-e2c1-451a-b377-30953c64b718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:26.619679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01080c5-e2c1-451a-b377-30953c64b718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:26.619956 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01080c5-e2c1-451a-b377-30953c64b718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:26.620225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01080c5-e2c1-451a-b377-30953c64b718 admin admin] PATH_INFO: `/v3/projects/3c53b0ab05544dec860358ffb05225b4/users/026a6d256c494b74988b2b200e936a11/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:26.623061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01080c5-e2c1-451a-b377-30953c64b718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c53b0ab05544dec860358ffb05225b4, user_id=026a6d256c494b74988b2b200e936a11, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:26.625663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01080c5-e2c1-451a-b377-30953c64b718 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:26.637459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1192/2357] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:24:26 2022] PUT /identity/v3/projects/3c53b0ab05544dec860358ffb05225b4/users/026a6d256c494b74988b2b200e936a11/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:24:36.593488 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:36.643312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:36.656130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89202ddc-5635-4a10-8437-337429a09872 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:36.656130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89202ddc-5635-4a10-8437-337429a09872 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:36.656130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89202ddc-5635-4a10-8437-337429a09872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:36.688107 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-89202ddc-5635-4a10-8437-337429a09872 None None] MFA Rules not processed for user `522765afd64c4018baebae36facfc3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:36.689366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-89202ddc-5635-4a10-8437-337429a09872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:36.815254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1166/2358] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:36.848346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf3a0-bc54-442f-b59d-ade7662b5f80 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:36.848633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf3a0-bc54-442f-b59d-ade7662b5f80 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:36.848906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf3a0-bc54-442f-b59d-ade7662b5f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:36.905783 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f98bf3a0-bc54-442f-b59d-ade7662b5f80 None None] MFA Rules not processed for user `026a6d256c494b74988b2b200e936a11`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:36.907023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f98bf3a0-bc54-442f-b59d-ade7662b5f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:37.050961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1193/2359] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:37.071601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89202ddc-5635-4a10-8437-337429a09872 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:37.074893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-89202ddc-5635-4a10-8437-337429a09872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:37.188275 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89202ddc-5635-4a10-8437-337429a09872 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:37.205487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc0c0c-33ce-4841-a661-4d7ae67b029c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b5ee9e4388d4bbabc7f1e9e3ca50b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:37.206594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc0c0c-33ce-4841-a661-4d7ae67b029c service glance] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:37.206889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc0c0c-33ce-4841-a661-4d7ae67b029c service glance] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:37.207304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc0c0c-33ce-4841-a661-4d7ae67b029c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:37.289252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc0c0c-33ce-4841-a661-4d7ae67b029c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:37.290740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc0c0c-33ce-4841-a661-4d7ae67b029c service glance] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:37.304307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1167/2360] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:37.664142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb114a6-6c16-4ec8-9b79-05be47a6addd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:37.664631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb114a6-6c16-4ec8-9b79-05be47a6addd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:37.664715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb114a6-6c16-4ec8-9b79-05be47a6addd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:37.692777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-deb114a6-6c16-4ec8-9b79-05be47a6addd None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:37.694726 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-deb114a6-6c16-4ec8-9b79-05be47a6addd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:37.797158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1194/2361] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:37.816856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc0c0c-33ce-4841-a661-4d7ae67b029c service glance] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:37.951772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:37.958474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc0c0c-33ce-4841-a661-4d7ae67b029c service glance] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:37.964786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7ecbc-ee85-4aca-930e-e061428b19c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:37.965920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7ecbc-ee85-4aca-930e-e061428b19c9 service swift] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:37.966376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7ecbc-ee85-4aca-930e-e061428b19c9 service swift] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:37.966809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7ecbc-ee85-4aca-930e-e061428b19c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:37.978626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7ecbc-ee85-4aca-930e-e061428b19c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:37.981927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7ecbc-ee85-4aca-930e-e061428b19c9 service swift] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:37.992578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1168/2362] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:24:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:24:47.778236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=59 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:48.009030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:24:50.689459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa1fcb-1cb0-46ac-b1ad-252c2d892610 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:50.689979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa1fcb-1cb0-46ac-b1ad-252c2d892610 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:50.690349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa1fcb-1cb0-46ac-b1ad-252c2d892610 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:50.692021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1195/2363] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:24:50.701104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6e147-b9f8-44fc-bd6d-a9f4f6b5b4ab None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:50.701670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6e147-b9f8-44fc-bd6d-a9f4f6b5b4ab None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:50.701670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6e147-b9f8-44fc-bd6d-a9f4f6b5b4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:50.737235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7bb6e147-b9f8-44fc-bd6d-a9f4f6b5b4ab None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:50.738582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7bb6e147-b9f8-44fc-bd6d-a9f4f6b5b4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:50.890646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1169/2364] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:50.902768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa1fcb-1cb0-46ac-b1ad-252c2d892610 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:50.904534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c2aa1fcb-1cb0-46ac-b1ad-252c2d892610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:50.932406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa1fcb-1cb0-46ac-b1ad-252c2d892610 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:50.937258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50614f-8440-4b7e-a0c8-880d140cf161 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b5ee9e4388d4bbabc7f1e9e3ca50b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:50.938004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50614f-8440-4b7e-a0c8-880d140cf161 service glance] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:50.938367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50614f-8440-4b7e-a0c8-880d140cf161 service glance] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:50.938721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50614f-8440-4b7e-a0c8-880d140cf161 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:50.954947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50614f-8440-4b7e-a0c8-880d140cf161 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:50.957082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50614f-8440-4b7e-a0c8-880d140cf161 service glance] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:50.971054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1196/2365] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:51.234018 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7e0f-98af-47ba-b74d-c0bd1ec6aac9 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:51.242934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7e0f-98af-47ba-b74d-c0bd1ec6aac9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:51.242934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7e0f-98af-47ba-b74d-c0bd1ec6aac9 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:51.242934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1170/2366] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:24:51.243273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d761c52-48b9-46fd-81c8-1441003b8cb6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:51.243273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d761c52-48b9-46fd-81c8-1441003b8cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:51.243273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d761c52-48b9-46fd-81c8-1441003b8cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:51.277685 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4d761c52-48b9-46fd-81c8-1441003b8cb6 None None] MFA Rules not processed for user `f7aa31ea778140189b041939e3f01aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:51.280893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4d761c52-48b9-46fd-81c8-1441003b8cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:51.365193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1197/2367] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:51.382172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f7e0f-98af-47ba-b74d-c0bd1ec6aac9 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:51.385097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-633f7e0f-98af-47ba-b74d-c0bd1ec6aac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:51.401376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f7e0f-98af-47ba-b74d-c0bd1ec6aac9 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:51.406162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f7b9e-399f-4958-90b5-f4f9b295224c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:51.406934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f7b9e-399f-4958-90b5-f4f9b295224c service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:51.407313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f7b9e-399f-4958-90b5-f4f9b295224c service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:51.407656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f7b9e-399f-4958-90b5-f4f9b295224c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:51.424333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f7b9e-399f-4958-90b5-f4f9b295224c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:51.426283 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f7b9e-399f-4958-90b5-f4f9b295224c service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:51.435650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1171/2368] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:24:51.523608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b1f0-9a8c-489d-ad12-03726b96ecd7 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:51.528568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b1f0-9a8c-489d-ad12-03726b96ecd7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:51.528568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5b1f0-9a8c-489d-ad12-03726b96ecd7 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:51.528568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1198/2369] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:24:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:24:51.534168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65bd8d-a6ff-43eb-91dc-df9a59d969fe None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:51.534456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65bd8d-a6ff-43eb-91dc-df9a59d969fe None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:51.534732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65bd8d-a6ff-43eb-91dc-df9a59d969fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:51.556712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8a65bd8d-a6ff-43eb-91dc-df9a59d969fe None None] MFA Rules not processed for user `cb2075e9fff3459881c13f81456ab198`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:51.559214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8a65bd8d-a6ff-43eb-91dc-df9a59d969fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:51.569082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1172/2370] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:24:51.586191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5b1f0-9a8c-489d-ad12-03726b96ecd7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:24:51.588149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2df5b1f0-9a8c-489d-ad12-03726b96ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:51.604749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5b1f0-9a8c-489d-ad12-03726b96ecd7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:24:51.609383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070d964-f830-4270-90d1-d1f3b9ab2a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:24:51.610229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d964-f830-4270-90d1-d1f3b9ab2a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:51.610681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d964-f830-4270-90d1-d1f3b9ab2a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:51.610986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d964-f830-4270-90d1-d1f3b9ab2a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:51.623944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070d964-f830-4270-90d1-d1f3b9ab2a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:24:51.625393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070d964-f830-4270-90d1-d1f3b9ab2a34 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:24:51.637606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1199/2371] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:24:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:24:51.894573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59733-aeac-4743-8900-b9308f65fbec None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:24:51.895247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59733-aeac-4743-8900-b9308f65fbec None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:24:51.895247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59733-aeac-4743-8900-b9308f65fbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:24:51.919437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-46f59733-aeac-4743-8900-b9308f65fbec None None] MFA Rules not processed for user `c62c3bfffc214c57a1f48b2779a615ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:24:51.921177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-46f59733-aeac-4743-8900-b9308f65fbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:24:51.932207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1173/2372] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:00.628868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726ab9c-a816-44eb-9c99-641b4a73c0df None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:00.629374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726ab9c-a816-44eb-9c99-641b4a73c0df None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:00.629757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726ab9c-a816-44eb-9c99-641b4a73c0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:00.654363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5726ab9c-a816-44eb-9c99-641b4a73c0df None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:00.655751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5726ab9c-a816-44eb-9c99-641b4a73c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:00.663555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1200/2373] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:00.677102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59733-aeac-4743-8900-b9308f65fbec None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:00.691962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59733-aeac-4743-8900-b9308f65fbec None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:00.698764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:00.699458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:00.699731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:00.700091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:00.709089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:00.710689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:00.720266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1174/2374] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:25:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:25:06.167331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddb1bf-044a-4d02-8a34-24810d661022 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:06.168065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddb1bf-044a-4d02-8a34-24810d661022 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:06.168065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddb1bf-044a-4d02-8a34-24810d661022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:06.187624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e3ddb1bf-044a-4d02-8a34-24810d661022 None None] MFA Rules not processed for user `b68f7224855a4cec81b664a2573d60f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:06.188813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e3ddb1bf-044a-4d02-8a34-24810d661022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:06.248335 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1201/2375] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:10.716526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=49 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:13.672004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:13.673456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:13.767243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6dac-5aff-4bb6-97b3-fba68c4b0323 service swift] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:13.772737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:13.773293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:13.774061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:13.774061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:13.775853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:13.788862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:13.790517 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:13.801929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1175/2376] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:25:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:25:15.077347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79f99b-a1e2-410a-a631-8e770a536756 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:15.077905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79f99b-a1e2-410a-a631-8e770a536756 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:15.078117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79f99b-a1e2-410a-a631-8e770a536756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:15.109406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-bb79f99b-a1e2-410a-a631-8e770a536756 None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:15.111855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bb79f99b-a1e2-410a-a631-8e770a536756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:15.122397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1202/2377] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:15.137154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:15.161936 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101046d-5720-482a-ab5a-723e2665df1e service placement] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:15.177542 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784355b-5c68-406f-b7d3-9018b68e6013 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:15.180403 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784355b-5c68-406f-b7d3-9018b68e6013 service swift] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:15.180761 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784355b-5c68-406f-b7d3-9018b68e6013 service swift] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:15.181047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784355b-5c68-406f-b7d3-9018b68e6013 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:15.201588 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784355b-5c68-406f-b7d3-9018b68e6013 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:15.205984 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784355b-5c68-406f-b7d3-9018b68e6013 service swift] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:15.240682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1176/2378] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:25:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:25:18.595653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8d1e4-697c-4e02-b77d-a163927aaf2e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:18.597173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8d1e4-697c-4e02-b77d-a163927aaf2e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:18.597688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8d1e4-697c-4e02-b77d-a163927aaf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:18.606492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=67 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:18.621947 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9ed8d1e4-697c-4e02-b77d-a163927aaf2e None None] MFA Rules not processed for user `10eaef565cf54b939bf78eba060f83d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:18.623424 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9ed8d1e4-697c-4e02-b77d-a163927aaf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:18.719202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1203/2379] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:25.220091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=38 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:28.695929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=24 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:32.465076 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18420-4019-4dd8-a223-96140a3c34bd None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:32.465614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18420-4019-4dd8-a223-96140a3c34bd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:32.465614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18420-4019-4dd8-a223-96140a3c34bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:32.490014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-49a18420-4019-4dd8-a223-96140a3c34bd None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:32.491816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-49a18420-4019-4dd8-a223-96140a3c34bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:32.501479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1177/2380] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:32.525812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8d1e4-697c-4e02-b77d-a163927aaf2e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:32.539082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8d1e4-697c-4e02-b77d-a163927aaf2e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:32.544593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b823de-e210-45b2-8f9d-eab38c5e1a1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:32.545237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b823de-e210-45b2-8f9d-eab38c5e1a1d service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:32.545518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b823de-e210-45b2-8f9d-eab38c5e1a1d service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:32.546098 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b823de-e210-45b2-8f9d-eab38c5e1a1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:32.554753 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b823de-e210-45b2-8f9d-eab38c5e1a1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:32.556679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b823de-e210-45b2-8f9d-eab38c5e1a1d service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:32.565655 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1204/2381] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:25:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:25:42.487860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:42.563619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:43.296805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e30c5e-e754-4362-9cd9-eb7e48262f6b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:43.297177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e30c5e-e754-4362-9cd9-eb7e48262f6b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:43.297558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e30c5e-e754-4362-9cd9-eb7e48262f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:43.323723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d9e30c5e-e754-4362-9cd9-eb7e48262f6b None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:43.325279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d9e30c5e-e754-4362-9cd9-eb7e48262f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:43.333187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1178/2382] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:43.350837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b823de-e210-45b2-8f9d-eab38c5e1a1d service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:43.371848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b823de-e210-45b2-8f9d-eab38c5e1a1d service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:43.377684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4f8c0-a1e6-4105-97ec-ea15be013224 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:43.378495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4f8c0-a1e6-4105-97ec-ea15be013224 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:43.378945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4f8c0-a1e6-4105-97ec-ea15be013224 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:43.379403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4f8c0-a1e6-4105-97ec-ea15be013224 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:43.394442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4f8c0-a1e6-4105-97ec-ea15be013224 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:43.397044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4f8c0-a1e6-4105-97ec-ea15be013224 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:43.408270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1205/2383] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:25:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:25:52.144011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aeceb-5f34-460d-a7dc-d813142bf5de None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:52.144483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aeceb-5f34-460d-a7dc-d813142bf5de None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:52.144587 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aeceb-5f34-460d-a7dc-d813142bf5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:52.164348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d53aeceb-5f34-460d-a7dc-d813142bf5de None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:52.165541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d53aeceb-5f34-460d-a7dc-d813142bf5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:52.173665 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1179/2384] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 12:25:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:52.186334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4f8c0-a1e6-4105-97ec-ea15be013224 service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:52.196805 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4f8c0-a1e6-4105-97ec-ea15be013224 service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:52.201974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0674888-f460-4d23-9c3c-0c7147a506bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:52.202555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0674888-f460-4d23-9c3c-0c7147a506bd service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:52.202835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0674888-f460-4d23-9c3c-0c7147a506bd service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:52.203139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0674888-f460-4d23-9c3c-0c7147a506bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:52.211259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0674888-f460-4d23-9c3c-0c7147a506bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:52.212826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0674888-f460-4d23-9c3c-0c7147a506bd service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:52.221620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1206/2385] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:25:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:25:55.093246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc8e68-f5f6-4d6a-be63-44cdffd1f709 None None] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:55.093739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc8e68-f5f6-4d6a-be63-44cdffd1f709 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:55.093832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc8e68-f5f6-4d6a-be63-44cdffd1f709 None None] PATH_INFO: `/` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:55.095156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1180/2386] 10.222.0.30 () {54 vars in 921 bytes} [Sun Aug 7 12:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 12:25:55.105340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd8f5f-de7b-41c7-9917-4c36e6bd0f82 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:55.105799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd8f5f-de7b-41c7-9917-4c36e6bd0f82 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:55.105953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd8f5f-de7b-41c7-9917-4c36e6bd0f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:55.129293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b4dd8f5f-de7b-41c7-9917-4c36e6bd0f82 None None] MFA Rules not processed for user `c3ee3d70937740b8adcb16ab57dc3031`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:55.130734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b4dd8f5f-de7b-41c7-9917-4c36e6bd0f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:55.139687 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1207/2387] 10.222.0.30 () {58 vars in 1018 bytes} [Sun Aug 7 12:25:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:25:55.154315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc8e68-f5f6-4d6a-be63-44cdffd1f709 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:55.168654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc8e68-f5f6-4d6a-be63-44cdffd1f709 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:55.173606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5f94c-4e23-4aeb-802d-f0900e5b12f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:55.174192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5f94c-4e23-4aeb-802d-f0900e5b12f3 service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:55.174468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5f94c-4e23-4aeb-802d-f0900e5b12f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:55.174735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5f94c-4e23-4aeb-802d-f0900e5b12f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:55.183462 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5f94c-4e23-4aeb-802d-f0900e5b12f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:55.185081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5f94c-4e23-4aeb-802d-f0900e5b12f3 service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:55.195101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1181/2388] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:25:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:25:55.252319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd8f5f-de7b-41c7-9917-4c36e6bd0f82 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:55.253720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b4dd8f5f-de7b-41c7-9917-4c36e6bd0f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:55.319364 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd8f5f-de7b-41c7-9917-4c36e6bd0f82 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:55.330084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:55.330742 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:55.331035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:55.331306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] PATH_INFO: `/v3/users/522765afd64c4018baebae36facfc3f3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:55.332569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=522765afd64c4018baebae36facfc3f3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:55.334592 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:55.522247 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522765afd64c4018baebae36facfc3f3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:55.529341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522765afd64c4018baebae36facfc3f3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:55.541607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522765afd64c4018baebae36facfc3f3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:55.542327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522765afd64c4018baebae36facfc3f3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:55.542937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a8571716-142b-429c-886f-d1c206373012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522765afd64c4018baebae36facfc3f3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:55.616498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1208/2389] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:25:55 2022] DELETE /identity/v3/users/522765afd64c4018baebae36facfc3f3 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:25:55.884496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5f94c-4e23-4aeb-802d-f0900e5b12f3 service placement] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:55.896213 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5f94c-4e23-4aeb-802d-f0900e5b12f3 service placement] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:55.952990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:55.953607 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:55.953884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:55.954153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] PATH_INFO: `/v3/projects/74e6b27298e34383acb124fd4fc56918` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:55.955301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e6b27298e34383acb124fd4fc56918)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:55.957171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:56.077394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e6b27298e34383acb124fd4fc56918', 'request_id': 'req-7e6b1746-8583-4245-933d-f38d0942e2d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:56.136613 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e6b27298e34383acb124fd4fc56918', 'request_id': 'req-7e6b1746-8583-4245-933d-f38d0942e2d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:56.142872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e6b27298e34383acb124fd4fc56918', 'request_id': 'req-7e6b1746-8583-4245-933d-f38d0942e2d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:56.154464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] The token cache is being invalidate because project 74e6b27298e34383acb124fd4fc56918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:25:56.154757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:56.155518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-7e6b1746-8583-4245-933d-f38d0942e2d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:25:56.156990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1182/2390] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:25:55 2022] DELETE /identity/v3/projects/74e6b27298e34383acb124fd4fc56918 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:25:56.161885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ea111-a23c-4bb2-b096-55e6a70eef5f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.162352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ea111-a23c-4bb2-b096-55e6a70eef5f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.162482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ea111-a23c-4bb2-b096-55e6a70eef5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.183161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5d6ea111-a23c-4bb2-b096-55e6a70eef5f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:56.184377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5d6ea111-a23c-4bb2-b096-55e6a70eef5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:56.227304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:56.244357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1209/2391] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:56.310576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b884ef5-01ca-4253-a002-f672a02b6d8f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.310948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b884ef5-01ca-4253-a002-f672a02b6d8f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.311278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b884ef5-01ca-4253-a002-f672a02b6d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.331448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7b884ef5-01ca-4253-a002-f672a02b6d8f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:56.332650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7b884ef5-01ca-4253-a002-f672a02b6d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:56.341083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1183/2392] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:56.346762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ea111-a23c-4bb2-b096-55e6a70eef5f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:56.355196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ea111-a23c-4bb2-b096-55e6a70eef5f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:56.359977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:56.360579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.360857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.361151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.361801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:25:56.362110 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:56.363830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:56.375630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1210/2393] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:25:56.379924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b884ef5-01ca-4253-a002-f672a02b6d8f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:56.389926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b884ef5-01ca-4253-a002-f672a02b6d8f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:56.394975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcd22d-a88e-4548-b8dc-373253aad9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:56.395590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcd22d-a88e-4548-b8dc-373253aad9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.395868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcd22d-a88e-4548-b8dc-373253aad9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.396158 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcd22d-a88e-4548-b8dc-373253aad9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.396752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcd22d-a88e-4548-b8dc-373253aad9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:56.398638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcd22d-a88e-4548-b8dc-373253aad9ff admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:56.501433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1184/2394] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:25:56 2022] POST /identity/v3/projects => generated 391 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:25:56.505488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:56.516793 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176acb3-9311-4ec1-aa7a-dbe52064c9d0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:56.583375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292ab7e-aec0-445f-8c41-2c5e2e3f4b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:56.584003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292ab7e-aec0-445f-8c41-2c5e2e3f4b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.584393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292ab7e-aec0-445f-8c41-2c5e2e3f4b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.584552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292ab7e-aec0-445f-8c41-2c5e2e3f4b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.585184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292ab7e-aec0-445f-8c41-2c5e2e3f4b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:56.587334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292ab7e-aec0-445f-8c41-2c5e2e3f4b06 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:56.688932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1211/2395] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:25:56 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:25:56.690183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcd22d-a88e-4548-b8dc-373253aad9ff admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:56.701573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcd22d-a88e-4548-b8dc-373253aad9ff admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:56.706231 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03801a-12ae-4064-a2c6-99be1af6f935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:56.711422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03801a-12ae-4064-a2c6-99be1af6f935 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.711422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03801a-12ae-4064-a2c6-99be1af6f935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.711422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03801a-12ae-4064-a2c6-99be1af6f935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.711422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03801a-12ae-4064-a2c6-99be1af6f935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:56.711422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03801a-12ae-4064-a2c6-99be1af6f935 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:56.719941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1185/2396] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:25:56.723979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292ab7e-aec0-445f-8c41-2c5e2e3f4b06 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:56.737468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292ab7e-aec0-445f-8c41-2c5e2e3f4b06 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:56.742306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a269da-c92d-4296-9db5-792d9ec5ff42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:56.742922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a269da-c92d-4296-9db5-792d9ec5ff42 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.743261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a269da-c92d-4296-9db5-792d9ec5ff42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.743539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a269da-c92d-4296-9db5-792d9ec5ff42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.744716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a269da-c92d-4296-9db5-792d9ec5ff42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:56.746422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a269da-c92d-4296-9db5-792d9ec5ff42 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:56.757461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1212/2397] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:25:56.762285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03801a-12ae-4064-a2c6-99be1af6f935 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:56.776787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03801a-12ae-4064-a2c6-99be1af6f935 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:56.781609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:56.782261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.782545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.782816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] PATH_INFO: `/v3/projects/186c37c5f9024c75949324b24c93f172/users/0c1ce22be3504da9a42cb0d07176269f/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.798413 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=186c37c5f9024c75949324b24c93f172, user_id=0c1ce22be3504da9a42cb0d07176269f, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:56.801194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:56.863739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:56.868599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1186/2398] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:25:56 2022] PUT /identity/v3/projects/186c37c5f9024c75949324b24c93f172/users/0c1ce22be3504da9a42cb0d07176269f/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:25:56.888370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b21d7-f4b9-4cfa-9450-ed2fb3873942 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.888765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b21d7-f4b9-4cfa-9450-ed2fb3873942 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.889042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b21d7-f4b9-4cfa-9450-ed2fb3873942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.899702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:56.901917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:56.920330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f97b21d7-f4b9-4cfa-9450-ed2fb3873942 None None] MFA Rules not processed for user `0c1ce22be3504da9a42cb0d07176269f`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:25:56.922642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f97b21d7-f4b9-4cfa-9450-ed2fb3873942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:56.963273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082b07a-eb6e-4980-a3a4-4d59c6eb8d44 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:56.967311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2ca09-b066-407a-a023-7e776b7d2c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:56.967918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2ca09-b066-407a-a023-7e776b7d2c40 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:56.968195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2ca09-b066-407a-a023-7e776b7d2c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:56.968463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2ca09-b066-407a-a023-7e776b7d2c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:56.970184 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-5fe2ca09-b066-407a-a023-7e776b7d2c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:56.998167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1213/2399] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:25:57.009565 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b21d7-f4b9-4cfa-9450-ed2fb3873942 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:25:57.010906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f97b21d7-f4b9-4cfa-9450-ed2fb3873942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:25:57.037249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2ca09-b066-407a-a023-7e776b7d2c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:57.038696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2ca09-b066-407a-a023-7e776b7d2c40 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:57.050162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1187/2400] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:25:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:25:57.056514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:25:57.068401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b21d7-f4b9-4cfa-9450-ed2fb3873942 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:25:57.072144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c857-5937-4aca-bbee-25b45c2310a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:25:57.072757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c857-5937-4aca-bbee-25b45c2310a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:25:57.073036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c857-5937-4aca-bbee-25b45c2310a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:25:57.073376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c857-5937-4aca-bbee-25b45c2310a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:25:57.082861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389c857-5937-4aca-bbee-25b45c2310a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:25:57.084317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389c857-5937-4aca-bbee-25b45c2310a5 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:25:57.094568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1214/2401] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:26:07.045489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=50 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:26:07.118937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:26:22.665446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4872a-f5b2-49ae-9780-33300bc522a0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:22.666013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4872a-f5b2-49ae-9780-33300bc522a0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:22.666013 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4872a-f5b2-49ae-9780-33300bc522a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:22.685399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3df4872a-f5b2-49ae-9780-33300bc522a0 None None] MFA Rules not processed for user `90dc7b751dd14c6f80205e709826423b`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:26:22.686595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3df4872a-f5b2-49ae-9780-33300bc522a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:22.741882 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1188/2402] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:26:23.222050 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c857-5937-4aca-bbee-25b45c2310a5 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:26:23.285944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c857-5937-4aca-bbee-25b45c2310a5 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:26:23.290489 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:26:23.291142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:23.291426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:23.291739 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] PATH_INFO: `/v3/users/0c1ce22be3504da9a42cb0d07176269f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:23.292989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c1ce22be3504da9a42cb0d07176269f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:26:23.294902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:26:23.515358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c1ce22be3504da9a42cb0d07176269f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:23.522241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c1ce22be3504da9a42cb0d07176269f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:23.535340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c1ce22be3504da9a42cb0d07176269f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:23.536085 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c1ce22be3504da9a42cb0d07176269f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:23.536690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c1ce22be3504da9a42cb0d07176269f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:23.598559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1215/2403] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:26:23 2022] DELETE /identity/v3/users/0c1ce22be3504da9a42cb0d07176269f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:26:23.602951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807f539-599b-4b74-90b6-20dd696c6cb5 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:23.603410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807f539-599b-4b74-90b6-20dd696c6cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:23.603691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807f539-599b-4b74-90b6-20dd696c6cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:23.622731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-b807f539-599b-4b74-90b6-20dd696c6cb5 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:26:23.623966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b807f539-599b-4b74-90b6-20dd696c6cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:23.683334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1189/2404] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:26:23.693025 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:26:23.694657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:23.755238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca38f-3067-4ed9-ba29-79157bfab416 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:26:23.758932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:26:23.759582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:23.759868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:23.760143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:23.770790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:26:23.772269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:26:23.782903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1216/2405] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:26:24.233293 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807f539-599b-4b74-90b6-20dd696c6cb5 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:26:24.235454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b807f539-599b-4b74-90b6-20dd696c6cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:24.263657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807f539-599b-4b74-90b6-20dd696c6cb5 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:26:24.268805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:26:24.271415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:24.271415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:24.271415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] PATH_INFO: `/v3/projects/186c37c5f9024c75949324b24c93f172` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:24.271415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=186c37c5f9024c75949324b24c93f172)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:26:24.273439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:26:24.386781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186c37c5f9024c75949324b24c93f172', 'request_id': 'req-2a01342a-279b-421d-ad25-df419aec4a05'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:24.447242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186c37c5f9024c75949324b24c93f172', 'request_id': 'req-2a01342a-279b-421d-ad25-df419aec4a05'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:24.453505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186c37c5f9024c75949324b24c93f172', 'request_id': 'req-2a01342a-279b-421d-ad25-df419aec4a05'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:24.465942 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] The token cache is being invalidate because project 186c37c5f9024c75949324b24c93f172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:26:24.466323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:24.467079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:24.468674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1190/2406] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:26:24 2022] DELETE /identity/v3/projects/186c37c5f9024c75949324b24c93f172 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:26:33.778398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:26:34.461591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:26:45.024139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:26:45.026393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:45.037422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e64a-f4dd-4a5c-918e-d2dc837542d8 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:26:45.086362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a59e08-ede3-41d6-b74f-6f5d81f4fae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:26:45.086965 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a59e08-ede3-41d6-b74f-6f5d81f4fae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:45.087894 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a59e08-ede3-41d6-b74f-6f5d81f4fae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:45.088249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a59e08-ede3-41d6-b74f-6f5d81f4fae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:45.090121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b1a59e08-ede3-41d6-b74f-6f5d81f4fae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:45.161866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a59e08-ede3-41d6-b74f-6f5d81f4fae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:26:45.163331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a59e08-ede3-41d6-b74f-6f5d81f4fae6 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:26:45.174054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1217/2407] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:26:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:26:46.609431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:26:46.610861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:46.624647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01342a-279b-421d-ad25-df419aec4a05 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:26:46.628478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:26:46.629111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:46.629386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:46.629660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:46.631808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:46.648692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:26:46.651481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:26:46.673215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1191/2408] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:26:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:26:54.219539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42480f84-894a-4fa3-a9f7-228da2ad0570 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:54.220032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42480f84-894a-4fa3-a9f7-228da2ad0570 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:54.221459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42480f84-894a-4fa3-a9f7-228da2ad0570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:54.239365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-42480f84-894a-4fa3-a9f7-228da2ad0570 None None] MFA Rules not processed for user `c8fb20cfc3b6479198059b4383f47527`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:26:54.240580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-42480f84-894a-4fa3-a9f7-228da2ad0570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:54.296276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1218/2409] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:26:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:26:56.665418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:26:58.180519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:26:58.182040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:58.247601 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d3ced-fe53-479e-9258-0e0f0ead68ba service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:26:58.252442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:26:58.253067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:58.253351 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:58.253628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] PATH_INFO: `/v3/users/b68f7224855a4cec81b664a2573d60f1` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:58.254774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68f7224855a4cec81b664a2573d60f1)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:26:58.256749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:26:58.427124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68f7224855a4cec81b664a2573d60f1'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:58.438859 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68f7224855a4cec81b664a2573d60f1'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:58.445739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68f7224855a4cec81b664a2573d60f1'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:58.447103 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68f7224855a4cec81b664a2573d60f1'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:58.447733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68f7224855a4cec81b664a2573d60f1'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:58.517216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1192/2410] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:26:58 2022] DELETE /identity/v3/users/b68f7224855a4cec81b664a2573d60f1 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:26:58.819791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42480f84-894a-4fa3-a9f7-228da2ad0570 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:26:58.831747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42480f84-894a-4fa3-a9f7-228da2ad0570 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:26:58.879892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:26:58.923254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:26:58.923907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:58.924309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:58.924625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] PATH_INFO: `/v3/projects/7bdb62eaaade46ffa205828d13dba7be` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:58.925906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bdb62eaaade46ffa205828d13dba7be)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:26:58.927972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:26:59.072054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bdb62eaaade46ffa205828d13dba7be', 'request_id': 'req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:59.160543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bdb62eaaade46ffa205828d13dba7be', 'request_id': 'req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:59.167207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bdb62eaaade46ffa205828d13dba7be', 'request_id': 'req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:59.178402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] The token cache is being invalidate because project 7bdb62eaaade46ffa205828d13dba7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:26:59.178693 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:59.179467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e0fed19b-8bc2-4215-ad7c-2e33a5bff173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:26:59.181782 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1219/2411] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:26:58 2022] DELETE /identity/v3/projects/7bdb62eaaade46ffa205828d13dba7be => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:26:59.400964 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:26:59.402355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:59.474788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db321f-d251-48e5-8c69-d044481ead77 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:26:59.488654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:26:59.541596 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:26:59.542218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:26:59.542500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:26:59.542778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:26:59.544620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:26:59.607035 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:26:59.608840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:26:59.618925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1193/2412] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:26:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:27:07.195354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d75b31-15c7-4469-8ec4-308ac39fc6a9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:07.195835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d75b31-15c7-4469-8ec4-308ac39fc6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:07.199188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d75b31-15c7-4469-8ec4-308ac39fc6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:07.216688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c8d75b31-15c7-4469-8ec4-308ac39fc6a9 None None] MFA Rules not processed for user `10eaef565cf54b939bf78eba060f83d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:07.217922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c8d75b31-15c7-4469-8ec4-308ac39fc6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:07.273066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1220/2413] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:07.585319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:07.588730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:07.644778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34441af0-463e-47d8-844b-1b3ad4b73bb2 service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:07.648375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:07.649186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:07.649466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:07.649737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:07.651465 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:07.660704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:07.662127 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:07.670654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1194/2414] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:08.143996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d75b31-15c7-4469-8ec4-308ac39fc6a9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:08.145458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c8d75b31-15c7-4469-8ec4-308ac39fc6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:08.222657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d75b31-15c7-4469-8ec4-308ac39fc6a9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:08.228366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:08.228976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:08.229259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:08.229537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:08.231278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:08.244257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:08.246010 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:08.256510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1221/2415] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:08.483180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:08.484593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:08.495860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abde77-3e3d-4caf-bb2b-a4a7d078acef service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:08.499421 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:08.500062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:08.500346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:08.500624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:08.502267 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:08.512084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:08.513527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:08.522188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1195/2416] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:11.273664 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:11.275310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:11.288140 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:11.332103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6b4c6-ede9-43d7-aec0-10b15aadecb0 service placement] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:11.335738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:11.336329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:11.336603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:11.336924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:11.338807 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:11.397350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:11.398785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:11.408727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1222/2417] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:13.037736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:13.040872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:13.059273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d00a4-6995-4dc5-8080-284009c28bbf service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:13.065382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:13.066031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:13.066307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:13.066820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:13.068589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:13.086277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:13.086277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:13.097842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1196/2418] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:21.404518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=43 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:23.091578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:29.618417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:29.620001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:29.636747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458d0c-98dd-4102-9745-34ce26cf5ced service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:29.643437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:29.644056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:29.644336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:29.644611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:29.646374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:29.659373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:29.660944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:29.671929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1223/2419] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:36.285603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:36.295230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402f4ad-2246-4bc4-ab11-2d1c6524d81e service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:36.298760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:36.299374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:36.299652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:36.299923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:36.301608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:36.363193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:36.364549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:36.374599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1197/2420] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:39.667290 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:39.965661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:39.967145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:39.981310 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3465e-3a49-48b7-bda3-1119e2b77901 service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:39.985775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:39.986386 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:39.986665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:39.986937 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] PATH_INFO: `/v3/users/90dc7b751dd14c6f80205e709826423b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:39.990681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90dc7b751dd14c6f80205e709826423b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:39.990681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:40.161045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90dc7b751dd14c6f80205e709826423b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.168229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90dc7b751dd14c6f80205e709826423b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.180799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90dc7b751dd14c6f80205e709826423b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.181477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90dc7b751dd14c6f80205e709826423b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.182094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08fbb0ca-9f2f-4a13-a3c0-5f05e2d2a586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90dc7b751dd14c6f80205e709826423b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.262259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1224/2421] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:27:39 2022] DELETE /identity/v3/users/90dc7b751dd14c6f80205e709826423b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:40.511894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:40.522171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52d15-cb7f-4857-b8a9-e9b5e0ad630d service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:40.582976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:40.583633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:40.583915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:40.584190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] PATH_INFO: `/v3/projects/bd31c40f797549d082e083324e285b6e` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:40.585369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd31c40f797549d082e083324e285b6e)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:40.587356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:40.698511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd31c40f797549d082e083324e285b6e', 'request_id': 'req-954c9239-489b-4874-b7e3-ea91d620d0a9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.750105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd31c40f797549d082e083324e285b6e', 'request_id': 'req-954c9239-489b-4874-b7e3-ea91d620d0a9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.756797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd31c40f797549d082e083324e285b6e', 'request_id': 'req-954c9239-489b-4874-b7e3-ea91d620d0a9'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.773482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] The token cache is being invalidate because project bd31c40f797549d082e083324e285b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:27:40.773744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.774431 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-954c9239-489b-4874-b7e3-ea91d620d0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:40.775957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1198/2422] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:27:40 2022] DELETE /identity/v3/projects/bd31c40f797549d082e083324e285b6e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:40.781024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca9b4-8868-4338-aeea-f88b3b3fff25 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:40.781546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca9b4-8868-4338-aeea-f88b3b3fff25 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:40.781645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca9b4-8868-4338-aeea-f88b3b3fff25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:40.801909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6dbca9b4-8868-4338-aeea-f88b3b3fff25 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:40.803224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6dbca9b4-8868-4338-aeea-f88b3b3fff25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:40.862497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1225/2423] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:40.870249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab981a-f3a7-4cf8-82e4-4ed659549a03 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:40.870562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab981a-f3a7-4cf8-82e4-4ed659549a03 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:40.870840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab981a-f3a7-4cf8-82e4-4ed659549a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:40.890519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-15ab981a-f3a7-4cf8-82e4-4ed659549a03 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:40.891748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-15ab981a-f3a7-4cf8-82e4-4ed659549a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:40.900093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1199/2424] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:40.904185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbca9b4-8868-4338-aeea-f88b3b3fff25 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:40.914303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbca9b4-8868-4338-aeea-f88b3b3fff25 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:40.918922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:40.919541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:40.919825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:40.920101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:40.920751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:27:40.921057 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:40.922651 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:40.934425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1226/2425] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:27:40.938475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab981a-f3a7-4cf8-82e4-4ed659549a03 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:40.947792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab981a-f3a7-4cf8-82e4-4ed659549a03 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:40.952386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92f013-a2b4-4418-be68-3122d606fdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:40.952963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f013-a2b4-4418-be68-3122d606fdff admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:40.953237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f013-a2b4-4418-be68-3122d606fdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:40.953505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f013-a2b4-4418-be68-3122d606fdff admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:40.954124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92f013-a2b4-4418-be68-3122d606fdff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:40.956047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92f013-a2b4-4418-be68-3122d606fdff admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:40.971817 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:41.076685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1200/2426] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:27:40 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:27:41.080756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:41.091541 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311888ff-3464-4a42-a41a-18dab6c961f6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:41.149603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc0e90-5f2f-4483-ad9c-65f946d52d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:41.150235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc0e90-5f2f-4483-ad9c-65f946d52d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:41.150518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc0e90-5f2f-4483-ad9c-65f946d52d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:41.150794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc0e90-5f2f-4483-ad9c-65f946d52d40 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:41.151466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc0e90-5f2f-4483-ad9c-65f946d52d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:41.153412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc0e90-5f2f-4483-ad9c-65f946d52d40 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:41.161269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:41.224179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1227/2427] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:27:41 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:27:41.228565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92f013-a2b4-4418-be68-3122d606fdff admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:41.238852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92f013-a2b4-4418-be68-3122d606fdff admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:41.243739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06be7ba-caad-47bd-821a-3743d27940ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:41.244329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06be7ba-caad-47bd-821a-3743d27940ec admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:41.244609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06be7ba-caad-47bd-821a-3743d27940ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:41.244883 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06be7ba-caad-47bd-821a-3743d27940ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:41.245526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06be7ba-caad-47bd-821a-3743d27940ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:41.247172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06be7ba-caad-47bd-821a-3743d27940ec admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:41.257397 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1201/2428] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:27:41.261161 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc0e90-5f2f-4483-ad9c-65f946d52d40 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:41.271609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc0e90-5f2f-4483-ad9c-65f946d52d40 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:41.277316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53606f3-2291-4ed8-94bb-95ad2f418f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:41.281888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53606f3-2291-4ed8-94bb-95ad2f418f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:41.281888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53606f3-2291-4ed8-94bb-95ad2f418f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:41.281888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53606f3-2291-4ed8-94bb-95ad2f418f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:41.281888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53606f3-2291-4ed8-94bb-95ad2f418f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:41.281888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53606f3-2291-4ed8-94bb-95ad2f418f4f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:41.291511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1228/2429] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:27:41.295716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06be7ba-caad-47bd-821a-3743d27940ec admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:41.305671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06be7ba-caad-47bd-821a-3743d27940ec admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:41.310370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:41.311056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:41.311361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:41.311660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] PATH_INFO: `/v3/projects/367d404c6c9c401f8a21d2acafb02309/users/ca84d98863254aad945864fbc9a5ccf5/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:41.327460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=367d404c6c9c401f8a21d2acafb02309, user_id=ca84d98863254aad945864fbc9a5ccf5, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:41.330098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:41.398788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1202/2430] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:27:41 2022] PUT /identity/v3/projects/367d404c6c9c401f8a21d2acafb02309/users/ca84d98863254aad945864fbc9a5ccf5/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:27:41.409977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c1093-eaad-47ac-ac06-4721abbbf837 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:41.410258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c1093-eaad-47ac-ac06-4721abbbf837 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:41.410534 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c1093-eaad-47ac-ac06-4721abbbf837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:41.437567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a07c1093-eaad-47ac-ac06-4721abbbf837 None None] MFA Rules not processed for user `ca84d98863254aad945864fbc9a5ccf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:41.438759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a07c1093-eaad-47ac-ac06-4721abbbf837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:41.501888 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1229/2431] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:41.514772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:41.516529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:41.589509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb2798-c6c7-4283-9d53-7883da09f44d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:41.594881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af00ccd-1f1d-4774-8694-b1893371af6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:41.595510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af00ccd-1f1d-4774-8694-b1893371af6f service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:41.595792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af00ccd-1f1d-4774-8694-b1893371af6f service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:41.596065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af00ccd-1f1d-4774-8694-b1893371af6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:41.603891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af00ccd-1f1d-4774-8694-b1893371af6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:41.605190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af00ccd-1f1d-4774-8694-b1893371af6f service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:41.613982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1203/2432] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:49.678652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c1093-eaad-47ac-ac06-4721abbbf837 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:49.740347 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c1093-eaad-47ac-ac06-4721abbbf837 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:49.745465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:49.746083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:49.751490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:49.751490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] PATH_INFO: `/v3/users/ca84d98863254aad945864fbc9a5ccf5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:49.751490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca84d98863254aad945864fbc9a5ccf5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:49.751490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:49.934808 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca84d98863254aad945864fbc9a5ccf5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:49.941861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca84d98863254aad945864fbc9a5ccf5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:49.953124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:49.960281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca84d98863254aad945864fbc9a5ccf5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:49.960281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca84d98863254aad945864fbc9a5ccf5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:49.960281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca84d98863254aad945864fbc9a5ccf5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:50.029949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1230/2433] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:27:49 2022] DELETE /identity/v3/users/ca84d98863254aad945864fbc9a5ccf5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:27:50.034478 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d6c6-58f9-4b76-8a09-423a8aa5f1a9 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:50.034995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d6c6-58f9-4b76-8a09-423a8aa5f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:50.034995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d6c6-58f9-4b76-8a09-423a8aa5f1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:50.054544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f102d6c6-58f9-4b76-8a09-423a8aa5f1a9 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:50.056279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f102d6c6-58f9-4b76-8a09-423a8aa5f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:50.126948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1204/2434] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:50.138502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:50.140268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:50.203762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b076f-2708-49f5-a14f-267e0405a86f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:50.207403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948670d-cdaa-4572-9255-f5084bed800c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:50.208065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948670d-cdaa-4572-9255-f5084bed800c service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:50.208348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948670d-cdaa-4572-9255-f5084bed800c service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:50.208624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948670d-cdaa-4572-9255-f5084bed800c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:50.218397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948670d-cdaa-4572-9255-f5084bed800c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:50.219887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948670d-cdaa-4572-9255-f5084bed800c service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:50.232631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1231/2435] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:27:50.734117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102d6c6-58f9-4b76-8a09-423a8aa5f1a9 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:50.735629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f102d6c6-58f9-4b76-8a09-423a8aa5f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:50.750430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102d6c6-58f9-4b76-8a09-423a8aa5f1a9 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:50.756919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:50.762190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:50.762190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:50.762190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] PATH_INFO: `/v3/projects/367d404c6c9c401f8a21d2acafb02309` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:50.762190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367d404c6c9c401f8a21d2acafb02309)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:50.762190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:50.864201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:50.870133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367d404c6c9c401f8a21d2acafb02309', 'request_id': 'req-789aec9d-506e-4c68-a37a-b9b3802f4116'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:50.975399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367d404c6c9c401f8a21d2acafb02309', 'request_id': 'req-789aec9d-506e-4c68-a37a-b9b3802f4116'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:50.981959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367d404c6c9c401f8a21d2acafb02309', 'request_id': 'req-789aec9d-506e-4c68-a37a-b9b3802f4116'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:50.993965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] The token cache is being invalidate because project 367d404c6c9c401f8a21d2acafb02309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:27:50.994279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:50.995034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-789aec9d-506e-4c68-a37a-b9b3802f4116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:50.996519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1205/2436] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:27:50 2022] DELETE /identity/v3/projects/367d404c6c9c401f8a21d2acafb02309 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:51.001453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954094f1-3d5e-4d97-ae87-a5a198f38ccc None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.001953 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954094f1-3d5e-4d97-ae87-a5a198f38ccc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.002062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954094f1-3d5e-4d97-ae87-a5a198f38ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.021481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-954094f1-3d5e-4d97-ae87-a5a198f38ccc None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:51.022729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-954094f1-3d5e-4d97-ae87-a5a198f38ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:51.082506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1232/2437] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:51.090395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0151-0091-4583-9584-ebf4dcb14348 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.090884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0151-0091-4583-9584-ebf4dcb14348 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.090989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0151-0091-4583-9584-ebf4dcb14348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.110842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1f5e0151-0091-4583-9584-ebf4dcb14348 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:51.112090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1f5e0151-0091-4583-9584-ebf4dcb14348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:51.120930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1206/2438] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:51.125204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954094f1-3d5e-4d97-ae87-a5a198f38ccc None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:51.134871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954094f1-3d5e-4d97-ae87-a5a198f38ccc None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:51.139703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:51.140291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.140570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.140846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.141488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:27:51.141809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:51.143394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:51.154927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1233/2439] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:27:51.159218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e0151-0091-4583-9584-ebf4dcb14348 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:51.168994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e0151-0091-4583-9584-ebf4dcb14348 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:51.173826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cece4b-8237-45fd-b41e-6f3ea8848ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:51.174424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cece4b-8237-45fd-b41e-6f3ea8848ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.174702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cece4b-8237-45fd-b41e-6f3ea8848ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.174975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cece4b-8237-45fd-b41e-6f3ea8848ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.175619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cece4b-8237-45fd-b41e-6f3ea8848ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:51.177523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cece4b-8237-45fd-b41e-6f3ea8848ce2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:51.294158 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1207/2440] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:27:51 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:27:51.298241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:51.308659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e9316-57e4-48ab-9d98-14343540e08c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:51.364764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1959aca-d106-4a07-b390-93f602923960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:51.365356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1959aca-d106-4a07-b390-93f602923960 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.365635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1959aca-d106-4a07-b390-93f602923960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.365908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1959aca-d106-4a07-b390-93f602923960 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.366532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1959aca-d106-4a07-b390-93f602923960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:51.368511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1959aca-d106-4a07-b390-93f602923960 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:51.475248 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1234/2441] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:27:51 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:27:51.480248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cece4b-8237-45fd-b41e-6f3ea8848ce2 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:51.491133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cece4b-8237-45fd-b41e-6f3ea8848ce2 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:51.496000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e845573-a3d4-4841-b39e-01228a38a194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:51.496680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e845573-a3d4-4841-b39e-01228a38a194 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.496963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e845573-a3d4-4841-b39e-01228a38a194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.497238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e845573-a3d4-4841-b39e-01228a38a194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.497911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e845573-a3d4-4841-b39e-01228a38a194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:51.499547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e845573-a3d4-4841-b39e-01228a38a194 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:51.509660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1208/2442] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:27:51.513678 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1959aca-d106-4a07-b390-93f602923960 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:51.523833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1959aca-d106-4a07-b390-93f602923960 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:51.528862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e07df-b98f-4e59-8e61-e8dad6afa8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:51.529454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e07df-b98f-4e59-8e61-e8dad6afa8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.529734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e07df-b98f-4e59-8e61-e8dad6afa8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.530009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e07df-b98f-4e59-8e61-e8dad6afa8c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.530656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e07df-b98f-4e59-8e61-e8dad6afa8c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:51.532272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e07df-b98f-4e59-8e61-e8dad6afa8c1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:51.533505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:51.544068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1235/2443] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:27:51.548387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e845573-a3d4-4841-b39e-01228a38a194 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:51.558497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e845573-a3d4-4841-b39e-01228a38a194 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:51.563200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:51.563839 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.564119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.564397 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] PATH_INFO: `/v3/projects/23dad836f82240c6b6f2fa85184b78ec/users/ab9ea03d83854d479f9e465cdcbb22c3/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.579546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23dad836f82240c6b6f2fa85184b78ec, user_id=ab9ea03d83854d479f9e465cdcbb22c3, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:51.582257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:51.650630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1209/2444] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:27:51 2022] PUT /identity/v3/projects/23dad836f82240c6b6f2fa85184b78ec/users/ab9ea03d83854d479f9e465cdcbb22c3/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:51.655950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7a50d-a630-43a1-993e-09dad04844ac None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.656269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7a50d-a630-43a1-993e-09dad04844ac None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.656551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7a50d-a630-43a1-993e-09dad04844ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.675519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a7b7a50d-a630-43a1-993e-09dad04844ac None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:51.676730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a7b7a50d-a630-43a1-993e-09dad04844ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:51.735075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1236/2445] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:51.744657 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:51.746330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:51.807314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc34d55-830f-494c-a701-c25900ccb5d4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:51.810823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0498076-39c0-4260-8d96-c80bf0cb69bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:51.811456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0498076-39c0-4260-8d96-c80bf0cb69bc service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:51.811739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0498076-39c0-4260-8d96-c80bf0cb69bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:51.812010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0498076-39c0-4260-8d96-c80bf0cb69bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:51.820771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0498076-39c0-4260-8d96-c80bf0cb69bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:51.823020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0498076-39c0-4260-8d96-c80bf0cb69bc service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:51.832347 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1210/2446] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:27:53.059630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f72dc-bc9e-4ad3-b93c-62a9005e6038 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:53.060167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f72dc-bc9e-4ad3-b93c-62a9005e6038 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:53.060167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f72dc-bc9e-4ad3-b93c-62a9005e6038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:53.095145 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ba1f72dc-bc9e-4ad3-b93c-62a9005e6038 None None] MFA Rules not processed for user `ab9ea03d83854d479f9e465cdcbb22c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:53.097034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ba1f72dc-bc9e-4ad3-b93c-62a9005e6038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:53.173500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1237/2447] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:53 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:53.177938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0498076-39c0-4260-8d96-c80bf0cb69bc service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:53.188244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0498076-39c0-4260-8d96-c80bf0cb69bc service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:53.192972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43825f4-7ee1-49b1-b5a5-0ac229d73b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:53.193620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43825f4-7ee1-49b1-b5a5-0ac229d73b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:53.193911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43825f4-7ee1-49b1-b5a5-0ac229d73b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:53.194186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43825f4-7ee1-49b1-b5a5-0ac229d73b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:53.194877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43825f4-7ee1-49b1-b5a5-0ac229d73b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:53.196915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43825f4-7ee1-49b1-b5a5-0ac229d73b1a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:53.327812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1211/2448] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:27:53 2022] POST /identity/v3/projects => generated 403 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:27:53.331943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f72dc-bc9e-4ad3-b93c-62a9005e6038 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:53.343060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f72dc-bc9e-4ad3-b93c-62a9005e6038 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:53.399650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170739d3-6378-401b-9b87-4c73d9499ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:53.400242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170739d3-6378-401b-9b87-4c73d9499ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:53.400520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170739d3-6378-401b-9b87-4c73d9499ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:53.400794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170739d3-6378-401b-9b87-4c73d9499ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:53.401404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170739d3-6378-401b-9b87-4c73d9499ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:53.403381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170739d3-6378-401b-9b87-4c73d9499ae8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:53.467454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1238/2449] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:27:53 2022] POST /identity/v3/users => generated 308 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:27:53.471361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43825f4-7ee1-49b1-b5a5-0ac229d73b1a admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:53.482104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43825f4-7ee1-49b1-b5a5-0ac229d73b1a admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:53.486608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b820cc3-4041-4d6a-a1ad-65d99599c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:53.487222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b820cc3-4041-4d6a-a1ad-65d99599c47f admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:53.487515 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b820cc3-4041-4d6a-a1ad-65d99599c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:53.487784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b820cc3-4041-4d6a-a1ad-65d99599c47f admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:53.488420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b820cc3-4041-4d6a-a1ad-65d99599c47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:53.490008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b820cc3-4041-4d6a-a1ad-65d99599c47f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:53.500518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1212/2450] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:27:53.504469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170739d3-6378-401b-9b87-4c73d9499ae8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:53.515182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170739d3-6378-401b-9b87-4c73d9499ae8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:53.519866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac47bc-5f8d-452e-88a9-5225fb0d618b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:53.520512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac47bc-5f8d-452e-88a9-5225fb0d618b admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:53.520795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac47bc-5f8d-452e-88a9-5225fb0d618b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:53.521071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac47bc-5f8d-452e-88a9-5225fb0d618b admin admin] PATH_INFO: `/v3/projects/2507c75d43f44359b749f87e0583cf61/users/dff676c0ef724b92aa76d8c6686efc73/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:53.534561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:53.537532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac47bc-5f8d-452e-88a9-5225fb0d618b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2507c75d43f44359b749f87e0583cf61, user_id=dff676c0ef724b92aa76d8c6686efc73, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:53.540878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac47bc-5f8d-452e-88a9-5225fb0d618b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:53.620972 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1239/2451] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:27:53 2022] PUT /identity/v3/projects/2507c75d43f44359b749f87e0583cf61/users/dff676c0ef724b92aa76d8c6686efc73/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:27:53.624889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b820cc3-4041-4d6a-a1ad-65d99599c47f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:53.634821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b820cc3-4041-4d6a-a1ad-65d99599c47f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:53.667174 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:53.691245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ac38-1674-45ef-b0ab-c31a43323c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:53.694218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ac38-1674-45ef-b0ab-c31a43323c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:53.694218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ac38-1674-45ef-b0ab-c31a43323c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:53.694218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ac38-1674-45ef-b0ab-c31a43323c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:53.694218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ac38-1674-45ef-b0ab-c31a43323c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:53.694646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ac38-1674-45ef-b0ab-c31a43323c03 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:53.704552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1213/2452] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:27:53.708625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac47bc-5f8d-452e-88a9-5225fb0d618b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:53.722164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac47bc-5f8d-452e-88a9-5225fb0d618b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:53.724543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236384c2-dc38-4173-a7f5-d45ff5d62654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:53.725206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236384c2-dc38-4173-a7f5-d45ff5d62654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:53.725472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236384c2-dc38-4173-a7f5-d45ff5d62654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:53.725749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236384c2-dc38-4173-a7f5-d45ff5d62654 admin admin] PATH_INFO: `/v3/projects/2507c75d43f44359b749f87e0583cf61/users/dff676c0ef724b92aa76d8c6686efc73/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:53.728600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236384c2-dc38-4173-a7f5-d45ff5d62654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2507c75d43f44359b749f87e0583cf61, user_id=dff676c0ef724b92aa76d8c6686efc73, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:53.731249 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236384c2-dc38-4173-a7f5-d45ff5d62654 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:53.741324 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1240/2453] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:27:53 2022] PUT /identity/v3/projects/2507c75d43f44359b749f87e0583cf61/users/dff676c0ef724b92aa76d8c6686efc73/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:55.285463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67132b-1f30-489f-a589-aac11237d9b5 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:55.286318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67132b-1f30-489f-a589-aac11237d9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:55.286836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67132b-1f30-489f-a589-aac11237d9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:55.320308 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2c67132b-1f30-489f-a589-aac11237d9b5 None None] MFA Rules not processed for user `dff676c0ef724b92aa76d8c6686efc73`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:55.322571 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2c67132b-1f30-489f-a589-aac11237d9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:55.394751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1214/2454] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:55 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:57.062933 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236384c2-dc38-4173-a7f5-d45ff5d62654 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:57.143845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236384c2-dc38-4173-a7f5-d45ff5d62654 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:57.148587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:57.149906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:57.150186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:57.150532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] PATH_INFO: `/v3/users/ab9ea03d83854d479f9e465cdcbb22c3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:57.151761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab9ea03d83854d479f9e465cdcbb22c3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:57.153686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:57.340647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab9ea03d83854d479f9e465cdcbb22c3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:57.347533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab9ea03d83854d479f9e465cdcbb22c3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:57.359930 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab9ea03d83854d479f9e465cdcbb22c3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:57.360630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab9ea03d83854d479f9e465cdcbb22c3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:57.361246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab9ea03d83854d479f9e465cdcbb22c3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:57.441121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1241/2455] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:27:57 2022] DELETE /identity/v3/users/ab9ea03d83854d479f9e465cdcbb22c3 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:57.747352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67132b-1f30-489f-a589-aac11237d9b5 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:57.762435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67132b-1f30-489f-a589-aac11237d9b5 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:57.840760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:57.841117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:57.841415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:57.841878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] PATH_INFO: `/v3/projects/23dad836f82240c6b6f2fa85184b78ec` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:57.843177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23dad836f82240c6b6f2fa85184b78ec)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:57.850482 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:57.972735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23dad836f82240c6b6f2fa85184b78ec', 'request_id': 'req-8cad48ea-c796-4fe9-b80e-9f56db627df0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.073265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23dad836f82240c6b6f2fa85184b78ec', 'request_id': 'req-8cad48ea-c796-4fe9-b80e-9f56db627df0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.073265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:58.077904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23dad836f82240c6b6f2fa85184b78ec', 'request_id': 'req-8cad48ea-c796-4fe9-b80e-9f56db627df0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.090756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] The token cache is being invalidate because project 23dad836f82240c6b6f2fa85184b78ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:27:58.091302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.092048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.094911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1215/2456] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:27:57 2022] DELETE /identity/v3/projects/23dad836f82240c6b6f2fa85184b78ec => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:58.098680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:58.103436 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:58.134307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9f8f6-8afe-44e4-ba19-5182f60e9bab admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:58.201825 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:58.207891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:58.208533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:58.208816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:58.209104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] PATH_INFO: `/v3/users/dff676c0ef724b92aa76d8c6686efc73` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:58.210447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff676c0ef724b92aa76d8c6686efc73)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:58.213003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:58.404395 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff676c0ef724b92aa76d8c6686efc73'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.413589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff676c0ef724b92aa76d8c6686efc73'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.426756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff676c0ef724b92aa76d8c6686efc73'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.427555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff676c0ef724b92aa76d8c6686efc73'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.428156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6aea757a-55f7-4d31-9366-b67d99358bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff676c0ef724b92aa76d8c6686efc73'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:58.492392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1242/2457] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:27:58 2022] DELETE /identity/v3/users/dff676c0ef724b92aa76d8c6686efc73 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:58.839572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:58.841071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:58.854981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad48ea-c796-4fe9-b80e-9f56db627df0 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:58.917232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:58.917884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:58.918164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:58.918439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] PATH_INFO: `/v3/projects/2507c75d43f44359b749f87e0583cf61` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:58.919575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2507c75d43f44359b749f87e0583cf61)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:58.921460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:59.037523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2507c75d43f44359b749f87e0583cf61', 'request_id': 'req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:59.111934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2507c75d43f44359b749f87e0583cf61', 'request_id': 'req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:59.119224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2507c75d43f44359b749f87e0583cf61', 'request_id': 'req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:59.131142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] The token cache is being invalidate because project 2507c75d43f44359b749f87e0583cf61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:27:59.131485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:59.132249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-09de9e6e-fa07-48f4-934d-eaa6ff5c0b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:27:59.134369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1216/2458] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:27:58 2022] DELETE /identity/v3/projects/2507c75d43f44359b749f87e0583cf61 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:59.140861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3480-837f-47ea-beb1-2af254554b04 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.141322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3480-837f-47ea-beb1-2af254554b04 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.141456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3480-837f-47ea-beb1-2af254554b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.161584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-551c3480-837f-47ea-beb1-2af254554b04 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:59.162756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-551c3480-837f-47ea-beb1-2af254554b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:59.235062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1243/2459] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:59.242924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93696-7acb-408d-8b46-cf52da63679a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.243282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93696-7acb-408d-8b46-cf52da63679a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.243557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93696-7acb-408d-8b46-cf52da63679a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.266743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-45b93696-7acb-408d-8b46-cf52da63679a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:59.268108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-45b93696-7acb-408d-8b46-cf52da63679a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:59.276801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1217/2460] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:59.281575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3480-837f-47ea-beb1-2af254554b04 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:59.293620 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3480-837f-47ea-beb1-2af254554b04 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:59.295889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:59.296495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.296767 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.297042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.297680 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:27:59.297982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:59.299600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:59.311679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1244/2461] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:27:59.316142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93696-7acb-408d-8b46-cf52da63679a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:59.325890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93696-7acb-408d-8b46-cf52da63679a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:59.330468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e39ef-118c-4ec1-a6ec-e52b4403cb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:59.331090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e39ef-118c-4ec1-a6ec-e52b4403cb67 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.331366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e39ef-118c-4ec1-a6ec-e52b4403cb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.331634 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e39ef-118c-4ec1-a6ec-e52b4403cb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.332243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e39ef-118c-4ec1-a6ec-e52b4403cb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:59.334160 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e39ef-118c-4ec1-a6ec-e52b4403cb67 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:59.456694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1218/2462] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:27:59 2022] POST /identity/v3/projects => generated 389 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:27:59.457691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:59.468259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0f27d-6a17-4dc7-bfe1-5d729db4dfaa admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:59.526250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17d41d-772f-44e9-9ab8-141d0ba3e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:59.526841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d41d-772f-44e9-9ab8-141d0ba3e5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.527155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d41d-772f-44e9-9ab8-141d0ba3e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.527430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17d41d-772f-44e9-9ab8-141d0ba3e5a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.528057 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17d41d-772f-44e9-9ab8-141d0ba3e5a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:59.530032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17d41d-772f-44e9-9ab8-141d0ba3e5a1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:59.540507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:27:59.639360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1245/2463] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:27:59 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:27:59.640527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e39ef-118c-4ec1-a6ec-e52b4403cb67 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:59.651555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e39ef-118c-4ec1-a6ec-e52b4403cb67 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:59.656356 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4631d0-6b49-4d68-8ea9-e0b6428a5365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:59.662222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4631d0-6b49-4d68-8ea9-e0b6428a5365 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.662222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4631d0-6b49-4d68-8ea9-e0b6428a5365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.662222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4631d0-6b49-4d68-8ea9-e0b6428a5365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.662222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4631d0-6b49-4d68-8ea9-e0b6428a5365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:59.662222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4631d0-6b49-4d68-8ea9-e0b6428a5365 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:59.671340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1219/2464] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:27:59.675786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17d41d-772f-44e9-9ab8-141d0ba3e5a1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:59.686124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17d41d-772f-44e9-9ab8-141d0ba3e5a1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:59.690898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4290dde-785f-416e-a298-cd63330a213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:59.691345 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290dde-785f-416e-a298-cd63330a213b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.691745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290dde-785f-416e-a298-cd63330a213b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.692027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4290dde-785f-416e-a298-cd63330a213b admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.692674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4290dde-785f-416e-a298-cd63330a213b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:59.694400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4290dde-785f-416e-a298-cd63330a213b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:59.706217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1246/2465] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:27:59.710676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4631d0-6b49-4d68-8ea9-e0b6428a5365 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:59.721195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4631d0-6b49-4d68-8ea9-e0b6428a5365 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:27:59.725929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f3689-5b51-4468-ae27-15bd98618151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:27:59.726568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f3689-5b51-4468-ae27-15bd98618151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.726850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f3689-5b51-4468-ae27-15bd98618151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.727716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f3689-5b51-4468-ae27-15bd98618151 admin admin] PATH_INFO: `/v3/projects/fdcb58d057f1466296ae788cc0378547/users/9dae6878d2f2439a80b33e032b982c34/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.742082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f3689-5b51-4468-ae27-15bd98618151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcb58d057f1466296ae788cc0378547, user_id=9dae6878d2f2439a80b33e032b982c34, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:27:59.744789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f3689-5b51-4468-ae27-15bd98618151 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:27:59.833225 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1220/2466] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:27:59 2022] PUT /identity/v3/projects/fdcb58d057f1466296ae788cc0378547/users/9dae6878d2f2439a80b33e032b982c34/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:27:59.846679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a69f7-157b-4584-b52c-1ed9b7555e0d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:27:59.846987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a69f7-157b-4584-b52c-1ed9b7555e0d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:27:59.847240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a69f7-157b-4584-b52c-1ed9b7555e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:27:59.874970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ec8a69f7-157b-4584-b52c-1ed9b7555e0d None None] MFA Rules not processed for user `9dae6878d2f2439a80b33e032b982c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:27:59.876199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ec8a69f7-157b-4584-b52c-1ed9b7555e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:27:59.953062 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1247/2467] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:27:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:27:59.957903 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f3689-5b51-4468-ae27-15bd98618151 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:27:59.975166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:00.035397 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f3689-5b51-4468-ae27-15bd98618151 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:00.041137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732713a-04cf-4e57-8c17-b7e63cc858c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:00.041734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732713a-04cf-4e57-8c17-b7e63cc858c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:00.042034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732713a-04cf-4e57-8c17-b7e63cc858c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:00.042455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732713a-04cf-4e57-8c17-b7e63cc858c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:00.043055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732713a-04cf-4e57-8c17-b7e63cc858c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:00.045340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732713a-04cf-4e57-8c17-b7e63cc858c1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:00.155801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1221/2468] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:27:59 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:28:00.159912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a69f7-157b-4584-b52c-1ed9b7555e0d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:00.174689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a69f7-157b-4584-b52c-1ed9b7555e0d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:00.248584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f733a8-c23a-4d73-953d-99a167b38d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:00.249199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f733a8-c23a-4d73-953d-99a167b38d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:00.249552 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f733a8-c23a-4d73-953d-99a167b38d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:00.249831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f733a8-c23a-4d73-953d-99a167b38d9e admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:00.250454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f733a8-c23a-4d73-953d-99a167b38d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:00.252422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f733a8-c23a-4d73-953d-99a167b38d9e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:00.356748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1248/2469] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:28:00 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:28:00.360561 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732713a-04cf-4e57-8c17-b7e63cc858c1 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:00.370943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732713a-04cf-4e57-8c17-b7e63cc858c1 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:00.375857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0167fc-14f3-40b1-8162-a70729570310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:00.376510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0167fc-14f3-40b1-8162-a70729570310 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:00.376934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0167fc-14f3-40b1-8162-a70729570310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:00.377144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0167fc-14f3-40b1-8162-a70729570310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:00.377792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0167fc-14f3-40b1-8162-a70729570310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:00.379438 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0167fc-14f3-40b1-8162-a70729570310 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:00.390685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1222/2470] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:28:00.394875 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f733a8-c23a-4d73-953d-99a167b38d9e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:00.405750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f733a8-c23a-4d73-953d-99a167b38d9e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:00.410353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50964-7153-4e04-a132-4d62c39cfe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:00.411048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50964-7153-4e04-a132-4d62c39cfe03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:00.413466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50964-7153-4e04-a132-4d62c39cfe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:00.413466 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50964-7153-4e04-a132-4d62c39cfe03 admin admin] PATH_INFO: `/v3/projects/cd5c3e8c20064d9abdd3075b76467c82/users/10bc4e637b1540eb8026840b1d00acc3/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:00.554785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c50964-7153-4e04-a132-4d62c39cfe03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd5c3e8c20064d9abdd3075b76467c82, user_id=10bc4e637b1540eb8026840b1d00acc3, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:00.557668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c50964-7153-4e04-a132-4d62c39cfe03 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:00.639151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1249/2471] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:28:00 2022] PUT /identity/v3/projects/cd5c3e8c20064d9abdd3075b76467c82/users/10bc4e637b1540eb8026840b1d00acc3/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:00.643139 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0167fc-14f3-40b1-8162-a70729570310 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:00.654045 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0167fc-14f3-40b1-8162-a70729570310 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:00.710781 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e579cf-a908-416d-aff3-a2f8c56dd8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:00.712137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e579cf-a908-416d-aff3-a2f8c56dd8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:00.712412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e579cf-a908-416d-aff3-a2f8c56dd8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:00.712682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e579cf-a908-416d-aff3-a2f8c56dd8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:00.713316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e579cf-a908-416d-aff3-a2f8c56dd8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:00.714919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e579cf-a908-416d-aff3-a2f8c56dd8a9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:00.724615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1223/2472] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:28:00.728671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50964-7153-4e04-a132-4d62c39cfe03 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:00.738786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50964-7153-4e04-a132-4d62c39cfe03 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:00.743421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:00.744060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:00.744332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:00.744606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] PATH_INFO: `/v3/projects/cd5c3e8c20064d9abdd3075b76467c82/users/10bc4e637b1540eb8026840b1d00acc3/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:00.746498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd5c3e8c20064d9abdd3075b76467c82, user_id=10bc4e637b1540eb8026840b1d00acc3, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:00.749257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:00.759951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1250/2473] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:28:00 2022] PUT /identity/v3/projects/cd5c3e8c20064d9abdd3075b76467c82/users/10bc4e637b1540eb8026840b1d00acc3/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:00.771772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d53540-1170-4b0c-8896-6f82cffb333a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:00.772057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d53540-1170-4b0c-8896-6f82cffb333a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:00.772334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d53540-1170-4b0c-8896-6f82cffb333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:00.801152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-54d53540-1170-4b0c-8896-6f82cffb333a None None] MFA Rules not processed for user `10bc4e637b1540eb8026840b1d00acc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:00.802367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-54d53540-1170-4b0c-8896-6f82cffb333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:00.870228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1224/2474] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:00.881009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:00.882671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:00.910288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:00.939918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ab4d9-d8dd-437e-8df4-e47fa48d9164 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:00.943544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4562d58-cdae-4ed3-a2df-305399273487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:00.944143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4562d58-cdae-4ed3-a2df-305399273487 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:00.944421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4562d58-cdae-4ed3-a2df-305399273487 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:00.944775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4562d58-cdae-4ed3-a2df-305399273487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:00.953636 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4562d58-cdae-4ed3-a2df-305399273487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:00.955077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4562d58-cdae-4ed3-a2df-305399273487 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:00.964829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1251/2475] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:01.134694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d53540-1170-4b0c-8896-6f82cffb333a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:01.146113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:01.202248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d53540-1170-4b0c-8896-6f82cffb333a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:01.206828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:01.207500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:01.207779 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:01.208055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] PATH_INFO: `/v3/users/9dae6878d2f2439a80b33e032b982c34` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:01.209188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dae6878d2f2439a80b33e032b982c34)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:01.211152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:01.471852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dae6878d2f2439a80b33e032b982c34'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:01.483533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dae6878d2f2439a80b33e032b982c34'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:01.490278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dae6878d2f2439a80b33e032b982c34'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:01.493639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dae6878d2f2439a80b33e032b982c34'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:01.493639 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dae6878d2f2439a80b33e032b982c34'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:01.578532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1225/2476] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:28:01 2022] DELETE /identity/v3/users/9dae6878d2f2439a80b33e032b982c34 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:01.583036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0552-3a7d-4901-9a5b-abd4bb7b2861 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:01.583519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0552-3a7d-4901-9a5b-abd4bb7b2861 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:01.584369 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0552-3a7d-4901-9a5b-abd4bb7b2861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:01.603768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-319c0552-3a7d-4901-9a5b-abd4bb7b2861 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:01.604954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-319c0552-3a7d-4901-9a5b-abd4bb7b2861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:01.665908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1252/2477] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:28:01.675680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:01.677136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:01.735521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e26b-b0f6-4419-b851-ad2a4c3a7611 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:01.739024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4067-1a81-4636-92da-e722eff506d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:01.739623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4067-1a81-4636-92da-e722eff506d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:01.739900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4067-1a81-4636-92da-e722eff506d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:01.740168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4067-1a81-4636-92da-e722eff506d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:01.749773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad4067-1a81-4636-92da-e722eff506d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:01.751250 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad4067-1a81-4636-92da-e722eff506d0 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:01.761738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1226/2478] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:02.198880 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c0552-3a7d-4901-9a5b-abd4bb7b2861 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:02.200324 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-319c0552-3a7d-4901-9a5b-abd4bb7b2861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:02.213595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c0552-3a7d-4901-9a5b-abd4bb7b2861 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:02.222020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:02.222659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:02.222938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:02.225125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] PATH_INFO: `/v3/projects/fdcb58d057f1466296ae788cc0378547` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:02.225125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdcb58d057f1466296ae788cc0378547)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:02.226434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:02.382678 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdcb58d057f1466296ae788cc0378547', 'request_id': 'req-737faba1-e980-4a67-95a8-23e010e2d06a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.546546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdcb58d057f1466296ae788cc0378547', 'request_id': 'req-737faba1-e980-4a67-95a8-23e010e2d06a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.552412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdcb58d057f1466296ae788cc0378547', 'request_id': 'req-737faba1-e980-4a67-95a8-23e010e2d06a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.563330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] The token cache is being invalidate because project fdcb58d057f1466296ae788cc0378547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:28:02.563668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.564238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.565688 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1253/2479] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:28:02 2022] DELETE /identity/v3/projects/fdcb58d057f1466296ae788cc0378547 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:02.569791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4067-1a81-4636-92da-e722eff506d0 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:02.579847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4067-1a81-4636-92da-e722eff506d0 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:02.609366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:02.635693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:02.636311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:02.636630 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:02.637105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] PATH_INFO: `/v3/users/10bc4e637b1540eb8026840b1d00acc3` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:02.639113 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bc4e637b1540eb8026840b1d00acc3)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:02.641088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:02.805170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bc4e637b1540eb8026840b1d00acc3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.816982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bc4e637b1540eb8026840b1d00acc3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.824344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bc4e637b1540eb8026840b1d00acc3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.825027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bc4e637b1540eb8026840b1d00acc3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.825636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ff6e0f08-06df-4133-b902-dfd3c20fd9b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bc4e637b1540eb8026840b1d00acc3'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:02.913855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1227/2480] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:28:02 2022] DELETE /identity/v3/users/10bc4e637b1540eb8026840b1d00acc3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:03.402188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:03.403951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:03.418917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737faba1-e980-4a67-95a8-23e010e2d06a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:03.472417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:03.482118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:03.482848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:03.483212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:03.483482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] PATH_INFO: `/v3/projects/cd5c3e8c20064d9abdd3075b76467c82` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:03.484602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd5c3e8c20064d9abdd3075b76467c82)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:03.492253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:03.583130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd5c3e8c20064d9abdd3075b76467c82', 'request_id': 'req-d68ab767-6c95-465a-8b61-f93c7e5e08b5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:03.671847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd5c3e8c20064d9abdd3075b76467c82', 'request_id': 'req-d68ab767-6c95-465a-8b61-f93c7e5e08b5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:03.677428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd5c3e8c20064d9abdd3075b76467c82', 'request_id': 'req-d68ab767-6c95-465a-8b61-f93c7e5e08b5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:03.688768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] The token cache is being invalidate because project cd5c3e8c20064d9abdd3075b76467c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:28:03.689129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:03.689702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d68ab767-6c95-465a-8b61-f93c7e5e08b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:03.691191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1254/2481] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:28:03 2022] DELETE /identity/v3/projects/cd5c3e8c20064d9abdd3075b76467c82 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:03.696792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41455e0-8878-4351-97d8-2e4cf4ca0568 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:03.697265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41455e0-8878-4351-97d8-2e4cf4ca0568 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:03.697402 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41455e0-8878-4351-97d8-2e4cf4ca0568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:03.717789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a41455e0-8878-4351-97d8-2e4cf4ca0568 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:03.718994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a41455e0-8878-4351-97d8-2e4cf4ca0568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:03.778809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1228/2482] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:03.786428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc590a4-9f95-4282-a82e-246f78b5ed73 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:03.786709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc590a4-9f95-4282-a82e-246f78b5ed73 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:03.786976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc590a4-9f95-4282-a82e-246f78b5ed73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:03.806354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fbc590a4-9f95-4282-a82e-246f78b5ed73 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:03.807622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fbc590a4-9f95-4282-a82e-246f78b5ed73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:03.815425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1255/2483] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:28:03.819764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41455e0-8878-4351-97d8-2e4cf4ca0568 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:03.830179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41455e0-8878-4351-97d8-2e4cf4ca0568 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:03.835078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:03.835688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:03.835967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:03.836851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:03.837506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:28:03.837808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:03.839495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:03.851324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1229/2484] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:28:03.855629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc590a4-9f95-4282-a82e-246f78b5ed73 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:03.864735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc590a4-9f95-4282-a82e-246f78b5ed73 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:03.869362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143bdd-9ea8-495d-8449-a5f89ba0fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:03.869942 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143bdd-9ea8-495d-8449-a5f89ba0fab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:03.870491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143bdd-9ea8-495d-8449-a5f89ba0fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:03.870491 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143bdd-9ea8-495d-8449-a5f89ba0fab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:03.871181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e143bdd-9ea8-495d-8449-a5f89ba0fab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:03.873052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e143bdd-9ea8-495d-8449-a5f89ba0fab8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:03.954421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1256/2485] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:28:03 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:28:03.957864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:03.968621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39017d75-49c0-42fe-9e27-bf8794f1f7a3 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:04.004723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:04.029085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c65037-c772-4fb7-b543-21a33763ce0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:04.029710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65037-c772-4fb7-b543-21a33763ce0b admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:04.030011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65037-c772-4fb7-b543-21a33763ce0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:04.030280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65037-c772-4fb7-b543-21a33763ce0b admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:04.030958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c65037-c772-4fb7-b543-21a33763ce0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:04.032898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c65037-c772-4fb7-b543-21a33763ce0b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:04.161853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1230/2486] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:28:03 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:28:04.166533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143bdd-9ea8-495d-8449-a5f89ba0fab8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:04.177022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143bdd-9ea8-495d-8449-a5f89ba0fab8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:04.181852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03ab64-8a56-4d0d-b8b0-852e002b26d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:04.183984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03ab64-8a56-4d0d-b8b0-852e002b26d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:04.183984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03ab64-8a56-4d0d-b8b0-852e002b26d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:04.183984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03ab64-8a56-4d0d-b8b0-852e002b26d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:04.183984 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03ab64-8a56-4d0d-b8b0-852e002b26d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:04.185233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03ab64-8a56-4d0d-b8b0-852e002b26d0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:04.198928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1257/2487] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:28:04.200318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c65037-c772-4fb7-b543-21a33763ce0b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:04.210209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c65037-c772-4fb7-b543-21a33763ce0b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:04.215949 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c92936-6b04-4e9a-bbee-1973d1e08580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:04.216554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92936-6b04-4e9a-bbee-1973d1e08580 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:04.216854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92936-6b04-4e9a-bbee-1973d1e08580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:04.217124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92936-6b04-4e9a-bbee-1973d1e08580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:04.217767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c92936-6b04-4e9a-bbee-1973d1e08580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:04.219407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c92936-6b04-4e9a-bbee-1973d1e08580 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:04.231803 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1231/2488] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:28:04.235780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03ab64-8a56-4d0d-b8b0-852e002b26d0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:04.245927 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03ab64-8a56-4d0d-b8b0-852e002b26d0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:04.250403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:04.251077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:04.251355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:04.251624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] PATH_INFO: `/v3/projects/e9b23069b8f24efe84a55a19df288b02/users/f6af3447e983451882713aeb6a91aec4/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:04.273144 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b23069b8f24efe84a55a19df288b02, user_id=f6af3447e983451882713aeb6a91aec4, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:04.276577 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:04.355882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1258/2489] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:28:04 2022] PUT /identity/v3/projects/e9b23069b8f24efe84a55a19df288b02/users/f6af3447e983451882713aeb6a91aec4/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:04.366200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84534fc-d531-4dd6-ab68-bdaa58d5add0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:04.366562 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84534fc-d531-4dd6-ab68-bdaa58d5add0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:04.366907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84534fc-d531-4dd6-ab68-bdaa58d5add0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:04.396857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c84534fc-d531-4dd6-ab68-bdaa58d5add0 None None] MFA Rules not processed for user `f6af3447e983451882713aeb6a91aec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:04.398187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c84534fc-d531-4dd6-ab68-bdaa58d5add0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:04.466430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1232/2490] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:04.478660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:04.479712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:04.565468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7cac5-7138-4f48-84a2-bed398366588 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:04.571083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff569960-6697-4038-b736-bea3047d6d86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:04.571921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569960-6697-4038-b736-bea3047d6d86 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:04.572255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569960-6697-4038-b736-bea3047d6d86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:04.572528 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569960-6697-4038-b736-bea3047d6d86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:04.581453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff569960-6697-4038-b736-bea3047d6d86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:04.582877 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff569960-6697-4038-b736-bea3047d6d86 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:04.593071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1259/2491] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:28:04.619717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84534fc-d531-4dd6-ab68-bdaa58d5add0 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:04.691951 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84534fc-d531-4dd6-ab68-bdaa58d5add0 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:04.696789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:04.697424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:04.697975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:04.697975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] PATH_INFO: `/v3/users/f6af3447e983451882713aeb6a91aec4` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:04.699170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6af3447e983451882713aeb6a91aec4)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:04.701156 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:04.858624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6af3447e983451882713aeb6a91aec4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:04.860004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:04.871965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6af3447e983451882713aeb6a91aec4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:04.878918 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6af3447e983451882713aeb6a91aec4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:04.879699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6af3447e983451882713aeb6a91aec4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:04.880318 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6af3447e983451882713aeb6a91aec4'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:04.978628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1233/2492] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:28:04 2022] DELETE /identity/v3/users/f6af3447e983451882713aeb6a91aec4 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:04.983076 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c923-bfd6-4b99-a418-6f85b609e56a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:04.983367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c923-bfd6-4b99-a418-6f85b609e56a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:04.983644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c923-bfd6-4b99-a418-6f85b609e56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:05.003337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a400c923-bfd6-4b99-a418-6f85b609e56a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:05.009243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a400c923-bfd6-4b99-a418-6f85b609e56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:05.017981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:05.063581 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1260/2493] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:05.074165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:05.075612 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:05.136485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e35f7-bd0a-4b2a-8ffc-15e14fd76f73 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:05.140475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9aef0-27be-44f1-8125-39097f92bda4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:05.141104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9aef0-27be-44f1-8125-39097f92bda4 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:05.141378 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9aef0-27be-44f1-8125-39097f92bda4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:05.141645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9aef0-27be-44f1-8125-39097f92bda4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:05.150782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9aef0-27be-44f1-8125-39097f92bda4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:05.152342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9aef0-27be-44f1-8125-39097f92bda4 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:05.161638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1234/2494] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:05.583026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c923-bfd6-4b99-a418-6f85b609e56a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:05.584830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a400c923-bfd6-4b99-a418-6f85b609e56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:05.609659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c923-bfd6-4b99-a418-6f85b609e56a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:05.615261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:05.615983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:05.616376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:05.616715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] PATH_INFO: `/v3/projects/e9b23069b8f24efe84a55a19df288b02` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:05.618009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b23069b8f24efe84a55a19df288b02)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:05.620160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:05.731175 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b23069b8f24efe84a55a19df288b02', 'request_id': 'req-fdb31979-3c31-4a44-a9f6-f63ee550496f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:05.789095 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b23069b8f24efe84a55a19df288b02', 'request_id': 'req-fdb31979-3c31-4a44-a9f6-f63ee550496f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:05.797312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b23069b8f24efe84a55a19df288b02', 'request_id': 'req-fdb31979-3c31-4a44-a9f6-f63ee550496f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:05.808212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] The token cache is being invalidate because project e9b23069b8f24efe84a55a19df288b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:28:05.809112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:05.809768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fdb31979-3c31-4a44-a9f6-f63ee550496f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:05.811527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1261/2495] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:28:05 2022] DELETE /identity/v3/projects/e9b23069b8f24efe84a55a19df288b02 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:05.816199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0c908-b04a-4079-9fd1-364cfb811487 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:05.819965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0c908-b04a-4079-9fd1-364cfb811487 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:05.819965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0c908-b04a-4079-9fd1-364cfb811487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:05.839739 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-65c0c908-b04a-4079-9fd1-364cfb811487 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:05.842952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-65c0c908-b04a-4079-9fd1-364cfb811487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:05.904125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1235/2496] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:05.912689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843eb42-50a6-446b-ac0e-132363b935ba None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:05.913075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843eb42-50a6-446b-ac0e-132363b935ba None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:05.913411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843eb42-50a6-446b-ac0e-132363b935ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:05.933578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0843eb42-50a6-446b-ac0e-132363b935ba None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:05.934779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0843eb42-50a6-446b-ac0e-132363b935ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:05.942777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1262/2497] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:05.946899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0c908-b04a-4079-9fd1-364cfb811487 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:05.957205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0c908-b04a-4079-9fd1-364cfb811487 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:05.961765 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:05.962363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:05.962645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:05.963119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:05.964040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:28:05.964348 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:05.965986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:05.977454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1236/2498] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:28:05.982196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843eb42-50a6-446b-ac0e-132363b935ba None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:05.991411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843eb42-50a6-446b-ac0e-132363b935ba None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:05.997046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78d53e-a6e0-4488-b518-a39643173021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:05.997660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78d53e-a6e0-4488-b518-a39643173021 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:05.997932 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78d53e-a6e0-4488-b518-a39643173021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:05.998195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78d53e-a6e0-4488-b518-a39643173021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:05.998802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78d53e-a6e0-4488-b518-a39643173021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:06.000746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78d53e-a6e0-4488-b518-a39643173021 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:06.113660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1263/2499] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:28:05 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:28:06.117891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:06.128636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8c301-6f9e-424e-bc05-c5978d3d9802 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:06.177496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:06.184526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5f23f-0bac-4568-9e68-490be8633c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:06.185126 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f23f-0bac-4568-9e68-490be8633c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:06.194801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f23f-0bac-4568-9e68-490be8633c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:06.194801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f23f-0bac-4568-9e68-490be8633c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:06.194801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5f23f-0bac-4568-9e68-490be8633c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:06.194801 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5f23f-0bac-4568-9e68-490be8633c5d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:06.295941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1237/2500] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:28:06 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:28:06.300111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78d53e-a6e0-4488-b518-a39643173021 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:06.311284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78d53e-a6e0-4488-b518-a39643173021 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:06.315998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8ddd-4b66-4ca0-aa9b-6a9bd1582d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:06.316614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ddd-4b66-4ca0-aa9b-6a9bd1582d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:06.316921 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ddd-4b66-4ca0-aa9b-6a9bd1582d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:06.317191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ddd-4b66-4ca0-aa9b-6a9bd1582d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:06.317826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df8ddd-4b66-4ca0-aa9b-6a9bd1582d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:06.319432 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df8ddd-4b66-4ca0-aa9b-6a9bd1582d5a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:06.329182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1264/2501] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:28:06.333123 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5f23f-0bac-4568-9e68-490be8633c5d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:06.343740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5f23f-0bac-4568-9e68-490be8633c5d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:06.348374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f111f-91ab-497a-99b0-f60d510986f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:06.348960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f111f-91ab-497a-99b0-f60d510986f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:06.349236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f111f-91ab-497a-99b0-f60d510986f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:06.349506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f111f-91ab-497a-99b0-f60d510986f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:06.350145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f111f-91ab-497a-99b0-f60d510986f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:06.359171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f111f-91ab-497a-99b0-f60d510986f1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:06.370263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1238/2502] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:28:06.374338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8ddd-4b66-4ca0-aa9b-6a9bd1582d5a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:06.384281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8ddd-4b66-4ca0-aa9b-6a9bd1582d5a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:06.389319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:06.389950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:06.390226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:06.390504 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] PATH_INFO: `/v3/projects/03d79614aa884e348e6c7378a9843211/users/5d407d7eff8e469490e21dcfc28baf0d/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:06.405614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d79614aa884e348e6c7378a9843211, user_id=5d407d7eff8e469490e21dcfc28baf0d, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:06.408433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:06.494952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1265/2503] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:28:06 2022] PUT /identity/v3/projects/03d79614aa884e348e6c7378a9843211/users/5d407d7eff8e469490e21dcfc28baf0d/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:06.505458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4392-98b8-4f26-8dd8-72b9faab98b2 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:06.505744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4392-98b8-4f26-8dd8-72b9faab98b2 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:06.506058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4392-98b8-4f26-8dd8-72b9faab98b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:06.536659 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-446c4392-98b8-4f26-8dd8-72b9faab98b2 None None] MFA Rules not processed for user `5d407d7eff8e469490e21dcfc28baf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:06.538899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-446c4392-98b8-4f26-8dd8-72b9faab98b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:06.599958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1239/2504] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:28:06.611233 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:06.612905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:06.655746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:06.669429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5dc55-d683-4026-91c7-d00dc423d4bf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:06.673088 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b5ee9e4388d4bbabc7f1e9e3ca50b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:06.673686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:06.673957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:06.674226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:06.683051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:06.684470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:06.693761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1266/2505] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:06.827461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb9e5c-7f31-4abd-b495-6c6f28cfdc71 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:06.827944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb9e5c-7f31-4abd-b495-6c6f28cfdc71 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:06.828041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb9e5c-7f31-4abd-b495-6c6f28cfdc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:06.847526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bceb9e5c-7f31-4abd-b495-6c6f28cfdc71 None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:06.848755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bceb9e5c-7f31-4abd-b495-6c6f28cfdc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:06.906547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1240/2506] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 12:28:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:07.060073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:07.061477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:07.141551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649569d2-5c1d-492b-9a16-89b95d75a67a service glance] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:07.146703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:07.147468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:07.147902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:07.148180 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:07.156856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:07.158426 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:07.166627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1267/2507] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:28:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:28:16.208751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb9e5c-7f31-4abd-b495-6c6f28cfdc71 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:16.210406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bceb9e5c-7f31-4abd-b495-6c6f28cfdc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:16.291920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb9e5c-7f31-4abd-b495-6c6f28cfdc71 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:16.297345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:16.297917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:16.298199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:16.298475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:16.300221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:16.312975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:16.314583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:16.319199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:16.326043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1241/2508] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:16.379224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:16.380781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:16.444872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f4e9b-e79b-4a25-b3cd-5f383010b091 service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:16.449869 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:16.450502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:16.450908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:16.451207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] PATH_INFO: `/v3/users/c62c3bfffc214c57a1f48b2779a615ac` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:16.452399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62c3bfffc214c57a1f48b2779a615ac)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:16.456537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:17.072081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62c3bfffc214c57a1f48b2779a615ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.080453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62c3bfffc214c57a1f48b2779a615ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.093381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62c3bfffc214c57a1f48b2779a615ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.094515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62c3bfffc214c57a1f48b2779a615ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.094722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62c3bfffc214c57a1f48b2779a615ac'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.096482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:17.188816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1268/2509] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:28:16 2022] DELETE /identity/v3/users/c62c3bfffc214c57a1f48b2779a615ac => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:17.433209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:17.445862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804ea1a-10b7-48fe-8bb9-0a137a8869e0 service placement] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:17.507449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:17.508162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:17.508441 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:17.508713 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] PATH_INFO: `/v3/projects/006bf6eb97a14d8c8c33f880b0bb80bc` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:17.509898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=006bf6eb97a14d8c8c33f880b0bb80bc)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:17.512433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:17.612610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006bf6eb97a14d8c8c33f880b0bb80bc', 'request_id': 'req-fd6fc160-77de-4914-af88-cda93f06c341'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.672290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006bf6eb97a14d8c8c33f880b0bb80bc', 'request_id': 'req-fd6fc160-77de-4914-af88-cda93f06c341'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.679196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006bf6eb97a14d8c8c33f880b0bb80bc', 'request_id': 'req-fd6fc160-77de-4914-af88-cda93f06c341'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.692117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] The token cache is being invalidate because project 006bf6eb97a14d8c8c33f880b0bb80bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:28:17.692410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.693066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:17.694571 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1242/2510] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:28:17 2022] DELETE /identity/v3/projects/006bf6eb97a14d8c8c33f880b0bb80bc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:17.698560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:17.700791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:17.717199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34458adc-2a44-4c01-851c-070ca54f4447 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:17.776389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:17.776946 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:17.777306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:17.778178 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] PATH_INFO: `/v3/users/026a6d256c494b74988b2b200e936a11` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:17.779383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=026a6d256c494b74988b2b200e936a11)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:17.781325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:17.998167 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026a6d256c494b74988b2b200e936a11'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.005054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026a6d256c494b74988b2b200e936a11'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.018004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026a6d256c494b74988b2b200e936a11'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.018702 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026a6d256c494b74988b2b200e936a11'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.019316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026a6d256c494b74988b2b200e936a11'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.083341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1269/2511] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:28:17 2022] DELETE /identity/v3/users/026a6d256c494b74988b2b200e936a11 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:28:18.215722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:18.217124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:18.281200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fc160-77de-4914-af88-cda93f06c341 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:18.323861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:18.331638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:18.332241 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:18.332583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:18.332818 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:18.334603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:18.367421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:18.368999 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:18.404279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:18.405727 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:18.415805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1243/2512] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:18.445462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01c4f2-7bd5-47f9-a2cd-8d4aabb4f3ac admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:18.451116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:18.451852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:18.452135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:18.452417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] PATH_INFO: `/v3/projects/3c53b0ab05544dec860358ffb05225b4` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:18.453546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c53b0ab05544dec860358ffb05225b4)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:18.455539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:18.471171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:18.565210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c53b0ab05544dec860358ffb05225b4', 'request_id': 'req-d3386722-29b3-457e-b403-28fb893b4ec2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.648418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c53b0ab05544dec860358ffb05225b4', 'request_id': 'req-d3386722-29b3-457e-b403-28fb893b4ec2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.654575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c53b0ab05544dec860358ffb05225b4', 'request_id': 'req-d3386722-29b3-457e-b403-28fb893b4ec2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.665751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] The token cache is being invalidate because project 3c53b0ab05544dec860358ffb05225b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:28:18.666039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.666681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d3386722-29b3-457e-b403-28fb893b4ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:18.668786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1270/2513] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:28:18 2022] DELETE /identity/v3/projects/3c53b0ab05544dec860358ffb05225b4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:24.309554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:24.311228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:24.392782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5ed-25da-438a-ade1-9ef9d347846f service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:24.463306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:24.463969 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:24.464254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:24.465180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:24.467354 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:24.481210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:24.482796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:24.493753 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1244/2514] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:24.982206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c603c-81c9-4ff6-b1b5-78cb014e671d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:24.982206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c603c-81c9-4ff6-b1b5-78cb014e671d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:24.982206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c603c-81c9-4ff6-b1b5-78cb014e671d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:25.001500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-585c603c-81c9-4ff6-b1b5-78cb014e671d None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:25.001500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-585c603c-81c9-4ff6-b1b5-78cb014e671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:25.065582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1271/2515] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:28:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:25.078159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:25.079879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:25.121010 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:25.174943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee7b9e-3479-4b8f-9fbc-724902b72e10 service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:25.182076 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:25.182771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:25.183164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:25.184020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:25.194539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:25.196313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:25.207808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1245/2516] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:28:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:28:29.800226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c603c-81c9-4ff6-b1b5-78cb014e671d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:29.801775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-585c603c-81c9-4ff6-b1b5-78cb014e671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:29.867363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c603c-81c9-4ff6-b1b5-78cb014e671d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:29.874736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb85d47-f96f-4d5d-9727-dd687e2cb1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:29.875403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb85d47-f96f-4d5d-9727-dd687e2cb1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:29.875683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb85d47-f96f-4d5d-9727-dd687e2cb1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:29.876023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb85d47-f96f-4d5d-9727-dd687e2cb1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:29.877868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1bb85d47-f96f-4d5d-9727-dd687e2cb1f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:29.960425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb85d47-f96f-4d5d-9727-dd687e2cb1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:29.961939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb85d47-f96f-4d5d-9727-dd687e2cb1f8 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:29.970035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:29.976443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1272/2517] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:35.202524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=22 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:44.020144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:44.021669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:44.085628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb767c-49b5-43bf-9086-e6749e8fa48e service swift] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:44.090246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:44.090867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:44.091180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:44.091460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] PATH_INFO: `/v3/users/5d407d7eff8e469490e21dcfc28baf0d` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:44.092581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d407d7eff8e469490e21dcfc28baf0d)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:44.095502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:44.287851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d407d7eff8e469490e21dcfc28baf0d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:44.300395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d407d7eff8e469490e21dcfc28baf0d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:44.307529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d407d7eff8e469490e21dcfc28baf0d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:44.308338 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d407d7eff8e469490e21dcfc28baf0d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:44.308947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d407d7eff8e469490e21dcfc28baf0d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:44.405458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1246/2518] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:28:44 2022] DELETE /identity/v3/users/5d407d7eff8e469490e21dcfc28baf0d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:44.411055 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f0ce9-b339-453a-bd23-e0279a5bea84 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:44.411584 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f0ce9-b339-453a-bd23-e0279a5bea84 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:44.411837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f0ce9-b339-453a-bd23-e0279a5bea84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:44.432500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b84f0ce9-b339-453a-bd23-e0279a5bea84 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:44.433709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b84f0ce9-b339-453a-bd23-e0279a5bea84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:44.493796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1273/2519] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:44.503871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:44.505282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:44.563054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd699902-aa60-489a-a9be-a68910fecb16 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:44.566653 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467cf36-66fc-4a47-a380-d0ca6a6afee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:44.567287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467cf36-66fc-4a47-a380-d0ca6a6afee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:44.567564 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467cf36-66fc-4a47-a380-d0ca6a6afee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:44.567833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467cf36-66fc-4a47-a380-d0ca6a6afee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:44.577022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467cf36-66fc-4a47-a380-d0ca6a6afee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:44.578624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467cf36-66fc-4a47-a380-d0ca6a6afee5 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:44.588496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1247/2520] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:45.049444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f0ce9-b339-453a-bd23-e0279a5bea84 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:45.053275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b84f0ce9-b339-453a-bd23-e0279a5bea84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:45.064108 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f0ce9-b339-453a-bd23-e0279a5bea84 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:45.068891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:45.069599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.069969 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.070428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] PATH_INFO: `/v3/projects/03d79614aa884e348e6c7378a9843211` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.071533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d79614aa884e348e6c7378a9843211)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:45.073573 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:45.170929 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d79614aa884e348e6c7378a9843211', 'request_id': 'req-03a79869-fee7-46cb-8e74-10c656edd00c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:45.233803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d79614aa884e348e6c7378a9843211', 'request_id': 'req-03a79869-fee7-46cb-8e74-10c656edd00c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:45.240026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d79614aa884e348e6c7378a9843211', 'request_id': 'req-03a79869-fee7-46cb-8e74-10c656edd00c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:45.251750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] The token cache is being invalidate because project 03d79614aa884e348e6c7378a9843211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:28:45.252037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:45.252674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03a79869-fee7-46cb-8e74-10c656edd00c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:28:45.254132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1274/2521] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:28:45 2022] DELETE /identity/v3/projects/03d79614aa884e348e6c7378a9843211 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:28:45.258978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06082551-a271-4b53-bc23-4238371c4621 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.259470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06082551-a271-4b53-bc23-4238371c4621 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.259608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06082551-a271-4b53-bc23-4238371c4621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.279593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-06082551-a271-4b53-bc23-4238371c4621 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:45.280795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-06082551-a271-4b53-bc23-4238371c4621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:45.297452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:45.339454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1248/2522] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:45.348189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4973f9-46ad-4fb3-ab6d-8bf94913db45 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.348475 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4973f9-46ad-4fb3-ab6d-8bf94913db45 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.348746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4973f9-46ad-4fb3-ab6d-8bf94913db45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.368289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cf4973f9-46ad-4fb3-ab6d-8bf94913db45 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:45.369482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cf4973f9-46ad-4fb3-ab6d-8bf94913db45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:45.377955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1275/2523] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:45.382002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06082551-a271-4b53-bc23-4238371c4621 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:45.392116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06082551-a271-4b53-bc23-4238371c4621 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:45.396683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:45.397273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.397547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.397815 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.398452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:28:45.398751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:45.400388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:45.412381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1249/2524] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:28:45.416506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4973f9-46ad-4fb3-ab6d-8bf94913db45 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:45.426099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4973f9-46ad-4fb3-ab6d-8bf94913db45 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:45.430601 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b230a-5f62-40ad-b81b-38be074498c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:45.431242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b230a-5f62-40ad-b81b-38be074498c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.431520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b230a-5f62-40ad-b81b-38be074498c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.431790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b230a-5f62-40ad-b81b-38be074498c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.432410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b230a-5f62-40ad-b81b-38be074498c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:45.434258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b230a-5f62-40ad-b81b-38be074498c7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:45.508958 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1276/2525] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:28:45 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:28:45.512795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:45.523210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bc246-5399-4c08-be6e-c3f36d5b91ea admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:45.580011 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f17c69-2104-4ac5-bc1a-b13c75bc1b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:45.580612 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f17c69-2104-4ac5-bc1a-b13c75bc1b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.580898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f17c69-2104-4ac5-bc1a-b13c75bc1b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.581172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f17c69-2104-4ac5-bc1a-b13c75bc1b32 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.581793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f17c69-2104-4ac5-bc1a-b13c75bc1b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:45.583758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f17c69-2104-4ac5-bc1a-b13c75bc1b32 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:45.682365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1250/2526] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:28:45 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:28:45.686416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b230a-5f62-40ad-b81b-38be074498c7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:45.696690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b230a-5f62-40ad-b81b-38be074498c7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:45.701359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcec59-7215-49ac-b597-24ad59166561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:45.701952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcec59-7215-49ac-b597-24ad59166561 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.702761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcec59-7215-49ac-b597-24ad59166561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.703179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcec59-7215-49ac-b597-24ad59166561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.703833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcec59-7215-49ac-b597-24ad59166561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:45.705409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcec59-7215-49ac-b597-24ad59166561 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:45.715817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1277/2527] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:28:45.720731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f17c69-2104-4ac5-bc1a-b13c75bc1b32 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:45.731247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f17c69-2104-4ac5-bc1a-b13c75bc1b32 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:45.736244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2bd94-0cfd-486b-a43b-e3717a36ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:45.736842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2bd94-0cfd-486b-a43b-e3717a36ba59 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.737121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2bd94-0cfd-486b-a43b-e3717a36ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.737389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2bd94-0cfd-486b-a43b-e3717a36ba59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.738049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2bd94-0cfd-486b-a43b-e3717a36ba59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:45.739699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2bd94-0cfd-486b-a43b-e3717a36ba59 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:45.752920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1251/2528] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:28:45.757255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcec59-7215-49ac-b597-24ad59166561 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:45.772193 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcec59-7215-49ac-b597-24ad59166561 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:45.777355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:45.778113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.778398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.778669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] PATH_INFO: `/v3/projects/df8016a4d76146b7a327002a04951ee2/users/1caf5d7efce94320ba032c1ef3f5ca52/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.795570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8016a4d76146b7a327002a04951ee2, user_id=1caf5d7efce94320ba032c1ef3f5ca52, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:45.798511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:45.895079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1278/2529] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:28:45 2022] PUT /identity/v3/projects/df8016a4d76146b7a327002a04951ee2/users/1caf5d7efce94320ba032c1ef3f5ca52/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:28:45.905209 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dcd95-1fe2-4ea6-ad91-199efceba1e5 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:45.906050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dcd95-1fe2-4ea6-ad91-199efceba1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:45.906323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dcd95-1fe2-4ea6-ad91-199efceba1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:45.946849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8f4dcd95-1fe2-4ea6-ad91-199efceba1e5 None None] MFA Rules not processed for user `1caf5d7efce94320ba032c1ef3f5ca52`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:45.948366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8f4dcd95-1fe2-4ea6-ad91-199efceba1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:46.037157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1252/2530] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:46.048785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:46.050455 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:46.072974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:46.117593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202ba00-0f98-4528-a6dd-4e19292e9114 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:46.119717 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:46.120486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:46.120886 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:46.121252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:46.131774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:46.133363 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:46.144288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1279/2531] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:47.973738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da1c0f-7a80-4572-8d55-d5eb875a7669 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:47.974279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da1c0f-7a80-4572-8d55-d5eb875a7669 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:47.974388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da1c0f-7a80-4572-8d55-d5eb875a7669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:47.992902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a8da1c0f-7a80-4572-8d55-d5eb875a7669 None None] MFA Rules not processed for user `1caf5d7efce94320ba032c1ef3f5ca52`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:47.994098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a8da1c0f-7a80-4572-8d55-d5eb875a7669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:48.001568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1253/2532] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:53.767777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:53.769182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:53.852096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be40d43-d417-44d2-90b7-06be307dfc66 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:53.854401 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:53.855474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:53.855757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:53.856035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:53.857822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:53.928769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:53.932046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:53.941520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1280/2533] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:28:57.987900 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:28:58.672359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b06316-007c-4ead-862f-e602c29b9500 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:58.672718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b06316-007c-4ead-862f-e602c29b9500 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:58.673069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b06316-007c-4ead-862f-e602c29b9500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:58.694816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-42b06316-007c-4ead-862f-e602c29b9500 None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:58.704248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-42b06316-007c-4ead-862f-e602c29b9500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:58.770602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1254/2534] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:28:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:28:58.783128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:28:58.785121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:58.865437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e0c1-3a2a-4ac7-838e-e39d889c58c6 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:28:58.870813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:28:58.871441 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:58.871724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:58.871992 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:58.881215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:28:58.882833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:28:58.891624 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1281/2535] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:28:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:28:59.757704 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b680c-0d5b-4e30-9304-b83cbcb2500b None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:28:59.758256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b680c-0d5b-4e30-9304-b83cbcb2500b None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:28:59.758377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b680c-0d5b-4e30-9304-b83cbcb2500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:28:59.793687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f42b680c-0d5b-4e30-9304-b83cbcb2500b None None] MFA Rules not processed for user `10eaef565cf54b939bf78eba060f83d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:28:59.794917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f42b680c-0d5b-4e30-9304-b83cbcb2500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:28:59.923122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1255/2536] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:29:00.824101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:00.829804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:00.847467 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:29:00.864974 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7b217-d881-4c3a-89ff-0afa948a6339 service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:00.866418 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:00.868077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:00.868427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:00.868627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:00.871276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:00.899164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:00.899164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:00.915428 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1282/2537] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:29:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:29:04.938245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b680c-0d5b-4e30-9304-b83cbcb2500b None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:04.940436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f42b680c-0d5b-4e30-9304-b83cbcb2500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:04.960577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b680c-0d5b-4e30-9304-b83cbcb2500b None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:04.967228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:04.967228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:04.967754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:04.968435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] PATH_INFO: `/v3/users/f3b0a65629074281bd0ac256bf4c5ce8` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:04.970199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b0a65629074281bd0ac256bf4c5ce8)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:04.972359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:05.162628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b0a65629074281bd0ac256bf4c5ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.177505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b0a65629074281bd0ac256bf4c5ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.184864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b0a65629074281bd0ac256bf4c5ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.185636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b0a65629074281bd0ac256bf4c5ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.187194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b0a65629074281bd0ac256bf4c5ce8'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.257380 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1256/2538] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:29:04 2022] DELETE /identity/v3/users/f3b0a65629074281bd0ac256bf4c5ce8 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:29:05.524228 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:05.543709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40839e3d-74b6-437f-b3c0-2b89e218a47a service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:05.611408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:05.612053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:05.612337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:05.612611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] PATH_INFO: `/v3/projects/c879759a492b427d99fea0ed4f410908` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:05.613769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c879759a492b427d99fea0ed4f410908)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:05.615706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:05.742128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c879759a492b427d99fea0ed4f410908', 'request_id': 'req-530f133f-ee29-41cb-9000-3ba980e7500b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.804781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c879759a492b427d99fea0ed4f410908', 'request_id': 'req-530f133f-ee29-41cb-9000-3ba980e7500b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.812107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c879759a492b427d99fea0ed4f410908', 'request_id': 'req-530f133f-ee29-41cb-9000-3ba980e7500b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.824195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] The token cache is being invalidate because project c879759a492b427d99fea0ed4f410908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:29:05.824580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.825312 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:05.826915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1283/2539] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:29:05 2022] DELETE /identity/v3/projects/c879759a492b427d99fea0ed4f410908 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:29:05.831030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:05.833213 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:05.846851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7927438-2141-49a3-8d97-143dfd8a00f6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:05.867684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:29:05.911069 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:05.911914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:05.912306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:05.912686 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] PATH_INFO: `/v3/users/f7b7421aa5a84af89e2ac77de7589333` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:05.913975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b7421aa5a84af89e2ac77de7589333)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:05.916063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:06.132724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b7421aa5a84af89e2ac77de7589333'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:06.164358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b7421aa5a84af89e2ac77de7589333'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:06.173026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b7421aa5a84af89e2ac77de7589333'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:06.174237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b7421aa5a84af89e2ac77de7589333'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:06.175053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0dd4c31d-ea57-428e-b3fb-da8317dc35e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b7421aa5a84af89e2ac77de7589333'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:06.240259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1257/2540] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:29:05 2022] DELETE /identity/v3/users/f7b7421aa5a84af89e2ac77de7589333 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:29:06.581896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:06.587174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:06.597107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f133f-ee29-41cb-9000-3ba980e7500b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:06.694228 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:06.695253 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:06.695775 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:06.696196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] PATH_INFO: `/v3/projects/2a41158c1e664ece8158abc22df41a8c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:06.697555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a41158c1e664ece8158abc22df41a8c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:06.700305 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:06.735860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:29:07.069715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a41158c1e664ece8158abc22df41a8c', 'request_id': 'req-61d6571a-e875-4c6d-a0ec-d0743b48946c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:07.148792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a41158c1e664ece8158abc22df41a8c', 'request_id': 'req-61d6571a-e875-4c6d-a0ec-d0743b48946c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:07.162799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a41158c1e664ece8158abc22df41a8c', 'request_id': 'req-61d6571a-e875-4c6d-a0ec-d0743b48946c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:07.178056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] The token cache is being invalidate because project 2a41158c1e664ece8158abc22df41a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:29:07.178350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:07.179072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:07.181128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1284/2541] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:29:06 2022] DELETE /identity/v3/projects/2a41158c1e664ece8158abc22df41a8c => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:29:15.828652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8bdac-c310-4a75-af36-6522fe26a270 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:15.828652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8bdac-c310-4a75-af36-6522fe26a270 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:15.828652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8bdac-c310-4a75-af36-6522fe26a270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:15.847830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-bed8bdac-c310-4a75-af36-6522fe26a270 None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:29:15.849217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bed8bdac-c310-4a75-af36-6522fe26a270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:15.916166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1258/2542] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 12:29:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:29:15.946240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:15.947748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:16.018164 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6571a-e875-4c6d-a0ec-d0743b48946c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:16.023372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9f5cf-5a1f-4ac6-bfd3-444aba4f6c7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:16.023976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9f5cf-5a1f-4ac6-bfd3-444aba4f6c7f service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:16.024259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9f5cf-5a1f-4ac6-bfd3-444aba4f6c7f service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:16.024533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9f5cf-5a1f-4ac6-bfd3-444aba4f6c7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:16.033566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9f5cf-5a1f-4ac6-bfd3-444aba4f6c7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:16.035186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9f5cf-5a1f-4ac6-bfd3-444aba4f6c7f service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:16.043461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1285/2543] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:29:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:29:18.460965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec5c8e-33e9-4981-92c6-7b4da57578e8 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:18.461615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec5c8e-33e9-4981-92c6-7b4da57578e8 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:18.461615 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec5c8e-33e9-4981-92c6-7b4da57578e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:18.481294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-48ec5c8e-33e9-4981-92c6-7b4da57578e8 None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:29:18.482523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-48ec5c8e-33e9-4981-92c6-7b4da57578e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:18.490595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1259/2544] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:29:18.503306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9f5cf-5a1f-4ac6-bfd3-444aba4f6c7f service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:18.513764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9f5cf-5a1f-4ac6-bfd3-444aba4f6c7f service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:18.519042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093c8af-ac7e-4aad-9f1e-f47f932e6924 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:18.519835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093c8af-ac7e-4aad-9f1e-f47f932e6924 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:18.520186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093c8af-ac7e-4aad-9f1e-f47f932e6924 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:18.520525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093c8af-ac7e-4aad-9f1e-f47f932e6924 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:18.528587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093c8af-ac7e-4aad-9f1e-f47f932e6924 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:18.530362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093c8af-ac7e-4aad-9f1e-f47f932e6924 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:18.538435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1286/2545] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:29:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:29:19.440404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6d4c9-2079-4d53-9fa3-cde5af408167 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:19.440710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6d4c9-2079-4d53-9fa3-cde5af408167 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:19.440990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6d4c9-2079-4d53-9fa3-cde5af408167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:19.460508 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-51b6d4c9-2079-4d53-9fa3-cde5af408167 None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:29:19.461716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-51b6d4c9-2079-4d53-9fa3-cde5af408167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:19.469037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1260/2546] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:29:19.481114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093c8af-ac7e-4aad-9f1e-f47f932e6924 service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:19.491666 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093c8af-ac7e-4aad-9f1e-f47f932e6924 service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:19.496517 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16972f-f9f2-49a3-ad46-c26bd2376ff8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:19.505901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16972f-f9f2-49a3-ad46-c26bd2376ff8 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:19.505901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16972f-f9f2-49a3-ad46-c26bd2376ff8 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:19.505901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16972f-f9f2-49a3-ad46-c26bd2376ff8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:19.506593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16972f-f9f2-49a3-ad46-c26bd2376ff8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:19.507576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16972f-f9f2-49a3-ad46-c26bd2376ff8 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:19.516602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1287/2547] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:29:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:29:20.261928 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8b030-a6af-4280-800a-f914f351fa04 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:20.262217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8b030-a6af-4280-800a-f914f351fa04 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:20.262550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8b030-a6af-4280-800a-f914f351fa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:20.282350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0fe8b030-a6af-4280-800a-f914f351fa04 None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:29:20.285087 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0fe8b030-a6af-4280-800a-f914f351fa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:20.291015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1261/2548] 10.222.0.30 () {58 vars in 1009 bytes} [Sun Aug 7 12:29:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:29:20.303778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16972f-f9f2-49a3-ad46-c26bd2376ff8 service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:20.314334 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16972f-f9f2-49a3-ad46-c26bd2376ff8 service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:20.319307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98e9dd-1b83-43ea-b096-cde0353e1239 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:20.319890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e9dd-1b83-43ea-b096-cde0353e1239 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:20.320172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e9dd-1b83-43ea-b096-cde0353e1239 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:20.320448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e9dd-1b83-43ea-b096-cde0353e1239 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:20.328437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98e9dd-1b83-43ea-b096-cde0353e1239 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:20.329991 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98e9dd-1b83-43ea-b096-cde0353e1239 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:20.337918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1288/2549] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:29:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:29:20.952503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7209c9-b48d-464a-ba00-ef5552c37fee None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:20.952986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7209c9-b48d-464a-ba00-ef5552c37fee None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:20.953100 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7209c9-b48d-464a-ba00-ef5552c37fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:20.971627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-cb7209c9-b48d-464a-ba00-ef5552c37fee None None] MFA Rules not processed for user `87a5db9a38544dc189b0c0d7bb205d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:29:20.972860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cb7209c9-b48d-464a-ba00-ef5552c37fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:20.979873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1262/2550] 10.222.0.30 () {58 vars in 967 bytes} [Sun Aug 7 12:29:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:29:20.992037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98e9dd-1b83-43ea-b096-cde0353e1239 service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:21.003237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98e9dd-1b83-43ea-b096-cde0353e1239 service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:21.010096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f9f7f-dcb9-424a-ae58-c5f0351e36db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a66a21436030409484b17f803d0e5903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:21.010691 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9f7f-dcb9-424a-ae58-c5f0351e36db service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:21.010970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9f7f-dcb9-424a-ae58-c5f0351e36db service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:21.011287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9f7f-dcb9-424a-ae58-c5f0351e36db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:21.020518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f9f7f-dcb9-424a-ae58-c5f0351e36db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:21.022080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f9f7f-dcb9-424a-ae58-c5f0351e36db service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:21.030670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1289/2551] 10.222.0.30 () {60 vars in 1384 bytes} [Sun Aug 7 12:29:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 12:29:23.806850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7209c9-b48d-464a-ba00-ef5552c37fee None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:23.808306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cb7209c9-b48d-464a-ba00-ef5552c37fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:23.830526 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:29:23.873861 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7209c9-b48d-464a-ba00-ef5552c37fee None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:23.878353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:23.879134 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:23.879439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:23.879712 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] PATH_INFO: `/v3/users/1caf5d7efce94320ba032c1ef3f5ca52` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:23.880870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1caf5d7efce94320ba032c1ef3f5ca52)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:23.882784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:24.068598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1caf5d7efce94320ba032c1ef3f5ca52'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.080374 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1caf5d7efce94320ba032c1ef3f5ca52'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.087055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1caf5d7efce94320ba032c1ef3f5ca52'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.087723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1caf5d7efce94320ba032c1ef3f5ca52'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.088319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1caf5d7efce94320ba032c1ef3f5ca52'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.152358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1263/2552] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:29:23 2022] DELETE /identity/v3/users/1caf5d7efce94320ba032c1ef3f5ca52 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:29:24.157659 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bcbde-18ab-47d2-b9e2-bdcfc9ed8b3e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:24.158156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bcbde-18ab-47d2-b9e2-bdcfc9ed8b3e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:24.158265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bcbde-18ab-47d2-b9e2-bdcfc9ed8b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:24.176848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b40bcbde-18ab-47d2-b9e2-bdcfc9ed8b3e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:29:24.178052 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b40bcbde-18ab-47d2-b9e2-bdcfc9ed8b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:24.221309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:29:24.238778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1290/2553] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:29:24.248681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:24.250146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:24.308075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36570643-c7ad-4bb8-af0e-7f11674f97e1 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:24.312375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a572-25a2-459e-b303-be54b61f89c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:24.312982 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a572-25a2-459e-b303-be54b61f89c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:24.313262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a572-25a2-459e-b303-be54b61f89c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:24.313536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a572-25a2-459e-b303-be54b61f89c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:24.329237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1a572-25a2-459e-b303-be54b61f89c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:24.329237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1a572-25a2-459e-b303-be54b61f89c8 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:24.334149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1264/2554] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:29:24.556673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bcbde-18ab-47d2-b9e2-bdcfc9ed8b3e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:24.558083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b40bcbde-18ab-47d2-b9e2-bdcfc9ed8b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:24.572858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bcbde-18ab-47d2-b9e2-bdcfc9ed8b3e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:24.578246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:24.579123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:24.579561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:24.579883 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] PATH_INFO: `/v3/projects/df8016a4d76146b7a327002a04951ee2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:24.581152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8016a4d76146b7a327002a04951ee2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:24.583165 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:24.704417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8016a4d76146b7a327002a04951ee2', 'request_id': 'req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.777833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8016a4d76146b7a327002a04951ee2', 'request_id': 'req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.784741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8016a4d76146b7a327002a04951ee2', 'request_id': 'req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.796890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] The token cache is being invalidate because project df8016a4d76146b7a327002a04951ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:29:24.797562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.798061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:29:24.799563 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1291/2555] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:29:24 2022] DELETE /identity/v3/projects/df8016a4d76146b7a327002a04951ee2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:29:34.330436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=70,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:29:34.792961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:29:57.267728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a572-25a2-459e-b303-be54b61f89c8 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:29:57.286706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a572-25a2-459e-b303-be54b61f89c8 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:29:57.343494 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:29:57.344085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:29:57.344366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:29:57.344640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:29:57.346373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:29:57.455236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:29:57.456595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:29:57.475648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1265/2556] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:01.206011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:01.211548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:01.268633 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768f41f-36d9-4a43-918c-bd3b1c2fdaa0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:01.273224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:01.273842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:01.274120 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:01.274393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] PATH_INFO: `/v3/users/c8fb20cfc3b6479198059b4383f47527` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:01.275538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fb20cfc3b6479198059b4383f47527)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:01.277423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:01.499073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fb20cfc3b6479198059b4383f47527'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:01.505855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fb20cfc3b6479198059b4383f47527'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:01.518878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fb20cfc3b6479198059b4383f47527'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:01.519595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fb20cfc3b6479198059b4383f47527'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:01.520246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-66e72a4a-1e6e-4655-b70c-9df63afaba5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fb20cfc3b6479198059b4383f47527'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:01.605206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1292/2557] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:30:01 2022] DELETE /identity/v3/users/c8fb20cfc3b6479198059b4383f47527 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:01.853050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:01.863775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8a340-fa4e-4e48-a4b1-6f51fbe8f817 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:01.895898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:01.922319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:01.922947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:01.923281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:01.923551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] PATH_INFO: `/v3/projects/547ebf661b1b48f289d734a71b494889` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:01.924721 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=547ebf661b1b48f289d734a71b494889)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:01.927419 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:02.045685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547ebf661b1b48f289d734a71b494889', 'request_id': 'req-fe02475a-3316-47dd-9bec-64d379214ede'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:02.128930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547ebf661b1b48f289d734a71b494889', 'request_id': 'req-fe02475a-3316-47dd-9bec-64d379214ede'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:02.135226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547ebf661b1b48f289d734a71b494889', 'request_id': 'req-fe02475a-3316-47dd-9bec-64d379214ede'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:02.147555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] The token cache is being invalidate because project 547ebf661b1b48f289d734a71b494889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:30:02.147851 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:02.148505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fe02475a-3316-47dd-9bec-64d379214ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:02.149986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1266/2558] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:30:01 2022] DELETE /identity/v3/projects/547ebf661b1b48f289d734a71b494889 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:02.155553 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385d55-73b0-4cd9-9de2-922dbfdb3021 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.155837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385d55-73b0-4cd9-9de2-922dbfdb3021 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.156112 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385d55-73b0-4cd9-9de2-922dbfdb3021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.212613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6a385d55-73b0-4cd9-9de2-922dbfdb3021 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:02.213926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6a385d55-73b0-4cd9-9de2-922dbfdb3021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:02.264109 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:02.274075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1293/2559] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:02.282577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d975f-cc98-45e1-ab13-f31d4d88fb84 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.282862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d975f-cc98-45e1-ab13-f31d4d88fb84 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.283176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d975f-cc98-45e1-ab13-f31d4d88fb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.303938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-2b4d975f-cc98-45e1-ab13-f31d4d88fb84 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:02.305143 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2b4d975f-cc98-45e1-ab13-f31d4d88fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:02.313752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1267/2560] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:02.317944 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a385d55-73b0-4cd9-9de2-922dbfdb3021 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:02.328194 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a385d55-73b0-4cd9-9de2-922dbfdb3021 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:02.332740 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:02.333330 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.333608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.333882 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.334524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:30:02.334830 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:02.336493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:02.348640 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1294/2561] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:30:02.352387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d975f-cc98-45e1-ab13-f31d4d88fb84 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:02.361455 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d975f-cc98-45e1-ab13-f31d4d88fb84 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:02.366012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899106e2-8d12-4be8-87ae-20eeb53cf5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:02.366599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899106e2-8d12-4be8-87ae-20eeb53cf5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.366874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899106e2-8d12-4be8-87ae-20eeb53cf5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.367175 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899106e2-8d12-4be8-87ae-20eeb53cf5a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.367797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899106e2-8d12-4be8-87ae-20eeb53cf5a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:02.369670 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899106e2-8d12-4be8-87ae-20eeb53cf5a9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:02.478422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1268/2562] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:02 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:30:02.482741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:02.492721 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc466101-caad-4683-9344-a73b5c96a0b0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:02.550222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035b7d-ed46-47cb-b27c-693dd4797ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:02.550823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035b7d-ed46-47cb-b27c-693dd4797ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.551134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035b7d-ed46-47cb-b27c-693dd4797ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.551410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035b7d-ed46-47cb-b27c-693dd4797ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.552023 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc035b7d-ed46-47cb-b27c-693dd4797ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:02.553922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc035b7d-ed46-47cb-b27c-693dd4797ea4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:02.642891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1295/2563] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:02 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:30:02.647193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899106e2-8d12-4be8-87ae-20eeb53cf5a9 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:02.657692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899106e2-8d12-4be8-87ae-20eeb53cf5a9 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:02.662266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135d554-8e52-442e-833e-0e6329bb078e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:02.662878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135d554-8e52-442e-833e-0e6329bb078e admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.663190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135d554-8e52-442e-833e-0e6329bb078e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.663466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135d554-8e52-442e-833e-0e6329bb078e admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.664157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135d554-8e52-442e-833e-0e6329bb078e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:02.665750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135d554-8e52-442e-833e-0e6329bb078e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:02.676282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1269/2564] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:02.680149 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035b7d-ed46-47cb-b27c-693dd4797ea4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:02.690712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035b7d-ed46-47cb-b27c-693dd4797ea4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:02.695450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ca577-d10d-4467-a6cd-3e90c04b03bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:02.696044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ca577-d10d-4467-a6cd-3e90c04b03bc admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.696322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ca577-d10d-4467-a6cd-3e90c04b03bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.696600 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ca577-d10d-4467-a6cd-3e90c04b03bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.697242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ca577-d10d-4467-a6cd-3e90c04b03bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:02.698835 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ca577-d10d-4467-a6cd-3e90c04b03bc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:02.709510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1296/2565] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:02.713633 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135d554-8e52-442e-833e-0e6329bb078e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:02.723917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135d554-8e52-442e-833e-0e6329bb078e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:02.728683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:02.729320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.729599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.729871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] PATH_INFO: `/v3/projects/a311cd62403a4fa285c48a145dd9ccfb/users/20f7cd1ed1214372a5c6f441fb98f799/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.744706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a311cd62403a4fa285c48a145dd9ccfb, user_id=20f7cd1ed1214372a5c6f441fb98f799, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:02.748057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:02.837230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1270/2566] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:02 2022] PUT /identity/v3/projects/a311cd62403a4fa285c48a145dd9ccfb/users/20f7cd1ed1214372a5c6f441fb98f799/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:02.842269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683289-8852-49af-ba48-ccbb1c530bc9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.842560 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683289-8852-49af-ba48-ccbb1c530bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.842841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683289-8852-49af-ba48-ccbb1c530bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:02.861803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b4683289-8852-49af-ba48-ccbb1c530bc9 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:02.863020 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b4683289-8852-49af-ba48-ccbb1c530bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:02.921770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1297/2567] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:02.931367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:02.933159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:02.989180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601dbf-40da-4721-b405-c18529140cba admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:02.992788 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:02.993382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:02.993661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:02.993937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:03.002862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:03.004325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:03.013930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1271/2568] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:08.636061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ef32-5d40-4f0f-85a3-46db719890c1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:08.636578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ef32-5d40-4f0f-85a3-46db719890c1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:08.636578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ef32-5d40-4f0f-85a3-46db719890c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:08.668338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d916ef32-5d40-4f0f-85a3-46db719890c1 None None] MFA Rules not processed for user `20f7cd1ed1214372a5c6f441fb98f799`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:08.670072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d916ef32-5d40-4f0f-85a3-46db719890c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:08.734484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:08.744495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1298/2569] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:08.760447 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:08.761876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:08.766232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:08.864289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678fae5-585b-4245-abe8-521eef312c3a service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:08.869823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:08.871874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:08.871874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:08.871874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:08.878881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:08.880504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:08.890064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1272/2570] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:11.250960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e8906-cdc3-4950-98e9-cecebefc5269 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:11.251533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e8906-cdc3-4950-98e9-cecebefc5269 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:11.251533 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e8906-cdc3-4950-98e9-cecebefc5269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:11.273070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8b5e8906-cdc3-4950-98e9-cecebefc5269 None None] MFA Rules not processed for user `20f7cd1ed1214372a5c6f441fb98f799`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:11.274372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8b5e8906-cdc3-4950-98e9-cecebefc5269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:11.281857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1299/2571] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:13.818265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:13.819924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:13.918043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b56647-1f89-48a2-a974-c411be67dd80 service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:13.923514 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:13.924409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:13.924690 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:13.925012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:13.926826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:13.951099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:13.951099 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:13.952028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1273/2572] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:21.266197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:23.947718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:29.631648 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e8906-cdc3-4950-98e9-cecebefc5269 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:29.633206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8b5e8906-cdc3-4950-98e9-cecebefc5269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:29.671812 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e8906-cdc3-4950-98e9-cecebefc5269 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:29.675589 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:29.676203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:29.676488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:29.676764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:29.678545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:29.694671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:29.696188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:29.708063 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1300/2573] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:33.464435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:33.464435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:33.479512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a703ef-a88d-49ce-aaf6-c30465546287 service placement] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:33.484464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:33.485133 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:33.485545 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:33.485825 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] PATH_INFO: `/v3/users/10eaef565cf54b939bf78eba060f83d0` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:33.486973 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10eaef565cf54b939bf78eba060f83d0)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:33.488956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:33.728854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10eaef565cf54b939bf78eba060f83d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:33.744043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10eaef565cf54b939bf78eba060f83d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:33.753565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10eaef565cf54b939bf78eba060f83d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:33.754305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10eaef565cf54b939bf78eba060f83d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:33.755884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f40b5d17-0b7e-4468-ae4c-93a925070ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10eaef565cf54b939bf78eba060f83d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:33.846864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1274/2574] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:30:33 2022] DELETE /identity/v3/users/10eaef565cf54b939bf78eba060f83d0 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:34.104135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:34.123416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8229e-a025-4de0-bbab-215a2d3ab6e8 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:34.195090 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:34.195699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.196127 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.196545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] PATH_INFO: `/v3/projects/555fce0b106b485db6a6800bdced08d2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.197920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=555fce0b106b485db6a6800bdced08d2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:34.199715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:34.322692 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '555fce0b106b485db6a6800bdced08d2', 'request_id': 'req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:34.387610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '555fce0b106b485db6a6800bdced08d2', 'request_id': 'req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:34.394900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '555fce0b106b485db6a6800bdced08d2', 'request_id': 'req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:34.410336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] The token cache is being invalidate because project 555fce0b106b485db6a6800bdced08d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:30:34.410801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:34.411579 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a87deefb-bc1f-4ccf-b8fe-fcccbca7d4d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:34.413154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1301/2575] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:30:34 2022] DELETE /identity/v3/projects/555fce0b106b485db6a6800bdced08d2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:34.418239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9458a-efaf-4ea3-bc00-9c3502d51c77 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.418530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9458a-efaf-4ea3-bc00-9c3502d51c77 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.418806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9458a-efaf-4ea3-bc00-9c3502d51c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.439501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-93a9458a-efaf-4ea3-bc00-9c3502d51c77 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:34.440889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-93a9458a-efaf-4ea3-bc00-9c3502d51c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:34.504177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1275/2576] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:34.512320 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1118-f70d-4f68-a35d-dbbe1539f0fb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.512652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1118-f70d-4f68-a35d-dbbe1539f0fb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.513893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1118-f70d-4f68-a35d-dbbe1539f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.533765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-17ec1118-f70d-4f68-a35d-dbbe1539f0fb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:34.534954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-17ec1118-f70d-4f68-a35d-dbbe1539f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:34.543051 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1302/2577] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:34.547631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9458a-efaf-4ea3-bc00-9c3502d51c77 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:34.558074 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9458a-efaf-4ea3-bc00-9c3502d51c77 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:34.563842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:34.564500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.564769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.565043 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.565692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:30:34.565994 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:34.567688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:34.580313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1276/2578] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:30:34.584686 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec1118-f70d-4f68-a35d-dbbe1539f0fb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:34.596446 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec1118-f70d-4f68-a35d-dbbe1539f0fb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:34.601546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17317d-4508-4b02-be5d-cd876bd5224d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:34.602172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17317d-4508-4b02-be5d-cd876bd5224d admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.602453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17317d-4508-4b02-be5d-cd876bd5224d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.602727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17317d-4508-4b02-be5d-cd876bd5224d admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.603421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17317d-4508-4b02-be5d-cd876bd5224d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:34.605318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17317d-4508-4b02-be5d-cd876bd5224d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:34.724931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1303/2579] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:34 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:34.729136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:34.740400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701b763-6d53-4444-b83a-7911c54eb79c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:34.801425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33017e7-1bf7-4876-a88d-86e6840fd497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:34.801963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33017e7-1bf7-4876-a88d-86e6840fd497 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.802239 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33017e7-1bf7-4876-a88d-86e6840fd497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.802507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33017e7-1bf7-4876-a88d-86e6840fd497 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.803316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33017e7-1bf7-4876-a88d-86e6840fd497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:34.805319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33017e7-1bf7-4876-a88d-86e6840fd497 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:34.813361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:34.882691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1277/2580] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:34 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:34.886829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17317d-4508-4b02-be5d-cd876bd5224d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:34.898209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17317d-4508-4b02-be5d-cd876bd5224d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:34.903587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611efa3-2a8d-465e-858b-ceb39ddd2c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:34.904192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611efa3-2a8d-465e-858b-ceb39ddd2c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.904470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611efa3-2a8d-465e-858b-ceb39ddd2c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.904741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611efa3-2a8d-465e-858b-ceb39ddd2c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.905411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611efa3-2a8d-465e-858b-ceb39ddd2c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:34.907089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611efa3-2a8d-465e-858b-ceb39ddd2c76 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:34.917996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1304/2581] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:34.926366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33017e7-1bf7-4876-a88d-86e6840fd497 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:34.937502 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33017e7-1bf7-4876-a88d-86e6840fd497 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:34.942672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956fef1-0da7-45e1-ae27-4bb059a9473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:34.943533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956fef1-0da7-45e1-ae27-4bb059a9473c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.943887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956fef1-0da7-45e1-ae27-4bb059a9473c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.944229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956fef1-0da7-45e1-ae27-4bb059a9473c admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.944955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956fef1-0da7-45e1-ae27-4bb059a9473c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:34.946784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956fef1-0da7-45e1-ae27-4bb059a9473c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:34.958029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1278/2582] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:34.962327 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611efa3-2a8d-465e-858b-ceb39ddd2c76 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:34.972755 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611efa3-2a8d-465e-858b-ceb39ddd2c76 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:34.977357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a435721-c815-4bc4-8963-10e66708449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:34.978003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a435721-c815-4bc4-8963-10e66708449a admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:34.978282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a435721-c815-4bc4-8963-10e66708449a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:34.978556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a435721-c815-4bc4-8963-10e66708449a admin admin] PATH_INFO: `/v3/projects/aca06d25bb334255b4ae79dc225cb326/users/8f4b4e086911470499b4a212660b02aa/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:34.993989 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a435721-c815-4bc4-8963-10e66708449a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca06d25bb334255b4ae79dc225cb326, user_id=8f4b4e086911470499b4a212660b02aa, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:34.996779 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a435721-c815-4bc4-8963-10e66708449a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:34.999728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:35.074004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1305/2583] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:34 2022] PUT /identity/v3/projects/aca06d25bb334255b4ae79dc225cb326/users/8f4b4e086911470499b4a212660b02aa/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:30:35.086119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9281d-e0de-4e93-b16a-c50a7f85ad4f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:35.088389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9281d-e0de-4e93-b16a-c50a7f85ad4f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:35.089162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9281d-e0de-4e93-b16a-c50a7f85ad4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:35.124774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-69f9281d-e0de-4e93-b16a-c50a7f85ad4f None None] MFA Rules not processed for user `8f4b4e086911470499b4a212660b02aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:35.126092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-69f9281d-e0de-4e93-b16a-c50a7f85ad4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:35.191537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1279/2584] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:35.195988 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a435721-c815-4bc4-8963-10e66708449a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:35.265555 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a435721-c815-4bc4-8963-10e66708449a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:35.270542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefedda-a01a-4bad-b327-34fbeb813c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:35.271197 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefedda-a01a-4bad-b327-34fbeb813c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:35.271469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefedda-a01a-4bad-b327-34fbeb813c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:35.271735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefedda-a01a-4bad-b327-34fbeb813c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:35.272371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefedda-a01a-4bad-b327-34fbeb813c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:35.274263 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefedda-a01a-4bad-b327-34fbeb813c82 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:35.377878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1306/2585] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:35 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:35.382190 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9281d-e0de-4e93-b16a-c50a7f85ad4f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:35.400777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9281d-e0de-4e93-b16a-c50a7f85ad4f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:35.457965 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357bc37-045d-43c5-812e-e045672e92e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:35.458572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357bc37-045d-43c5-812e-e045672e92e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:35.458847 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357bc37-045d-43c5-812e-e045672e92e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:35.459152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357bc37-045d-43c5-812e-e045672e92e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:35.459778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357bc37-045d-43c5-812e-e045672e92e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:35.461682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357bc37-045d-43c5-812e-e045672e92e8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:35.549424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1280/2586] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:35 2022] POST /identity/v3/users => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:35.553444 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefedda-a01a-4bad-b327-34fbeb813c82 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:35.565449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefedda-a01a-4bad-b327-34fbeb813c82 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:35.570044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f4765-6fe6-436f-bc45-f91fb761cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:35.570639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f4765-6fe6-436f-bc45-f91fb761cce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:35.570920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f4765-6fe6-436f-bc45-f91fb761cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:35.571231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f4765-6fe6-436f-bc45-f91fb761cce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:35.571878 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f4765-6fe6-436f-bc45-f91fb761cce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:35.573461 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f4765-6fe6-436f-bc45-f91fb761cce4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:35.584580 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1307/2587] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:35.588943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357bc37-045d-43c5-812e-e045672e92e8 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:35.599277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357bc37-045d-43c5-812e-e045672e92e8 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:35.603998 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb401ab-0b81-44b5-8efc-d18abe695f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:35.604715 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb401ab-0b81-44b5-8efc-d18abe695f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:35.605003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb401ab-0b81-44b5-8efc-d18abe695f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:35.605279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb401ab-0b81-44b5-8efc-d18abe695f65 admin admin] PATH_INFO: `/v3/projects/c1cbf892823c42eb844fda97cb3d6b9e/users/7b6d19ed4a074a7b991ee26aac7cc891/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:35.620957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb401ab-0b81-44b5-8efc-d18abe695f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1cbf892823c42eb844fda97cb3d6b9e, user_id=7b6d19ed4a074a7b991ee26aac7cc891, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:35.624430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb401ab-0b81-44b5-8efc-d18abe695f65 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:35.699227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1281/2588] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:35 2022] PUT /identity/v3/projects/c1cbf892823c42eb844fda97cb3d6b9e/users/7b6d19ed4a074a7b991ee26aac7cc891/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:30:35.703450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f4765-6fe6-436f-bc45-f91fb761cce4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:35.714039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f4765-6fe6-436f-bc45-f91fb761cce4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:35.776158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8412a-0d37-4f59-b278-f7eea6690040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:35.776902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8412a-0d37-4f59-b278-f7eea6690040 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:35.777423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8412a-0d37-4f59-b278-f7eea6690040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:35.777892 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8412a-0d37-4f59-b278-f7eea6690040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:35.778769 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8412a-0d37-4f59-b278-f7eea6690040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:35.780681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8412a-0d37-4f59-b278-f7eea6690040 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:35.790955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1308/2589] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:35.795867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb401ab-0b81-44b5-8efc-d18abe695f65 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:35.806479 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb401ab-0b81-44b5-8efc-d18abe695f65 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:35.811870 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:35.812524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:35.812814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:35.813093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] PATH_INFO: `/v3/projects/c1cbf892823c42eb844fda97cb3d6b9e/users/7b6d19ed4a074a7b991ee26aac7cc891/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:35.815736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1cbf892823c42eb844fda97cb3d6b9e, user_id=7b6d19ed4a074a7b991ee26aac7cc891, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:35.820407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:35.832458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1282/2590] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:35 2022] PUT /identity/v3/projects/c1cbf892823c42eb844fda97cb3d6b9e/users/7b6d19ed4a074a7b991ee26aac7cc891/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:35.842920 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e342b52-ca36-4572-8ce0-739345f398bb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:35.843269 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e342b52-ca36-4572-8ce0-739345f398bb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:35.843545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e342b52-ca36-4572-8ce0-739345f398bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:35.874831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2e342b52-ca36-4572-8ce0-739345f398bb None None] MFA Rules not processed for user `7b6d19ed4a074a7b991ee26aac7cc891`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:35.876081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2e342b52-ca36-4572-8ce0-739345f398bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:35.928005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:35.946284 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1309/2591] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:30:35.957613 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:35.959454 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:35.985457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:36.051101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981197c6-bad8-4c9e-bc24-79aac3e0dca5 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:36.055685 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b296a-78d9-4a93-8f19-e2635e1fca49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:36.056296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b296a-78d9-4a93-8f19-e2635e1fca49 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:36.056576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b296a-78d9-4a93-8f19-e2635e1fca49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:36.056855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b296a-78d9-4a93-8f19-e2635e1fca49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:36.067732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b296a-78d9-4a93-8f19-e2635e1fca49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:36.069159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b296a-78d9-4a93-8f19-e2635e1fca49 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:36.079487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1283/2592] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:36.149703 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e342b52-ca36-4572-8ce0-739345f398bb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:36.216483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e342b52-ca36-4572-8ce0-739345f398bb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:36.221282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:36.221909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:36.222214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:36.222487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] PATH_INFO: `/v3/users/8f4b4e086911470499b4a212660b02aa` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:36.223625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4b4e086911470499b4a212660b02aa)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:36.225604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:36.389857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4b4e086911470499b4a212660b02aa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:36.396722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4b4e086911470499b4a212660b02aa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:36.408975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4b4e086911470499b4a212660b02aa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:36.409745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4b4e086911470499b4a212660b02aa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:36.410361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4b4e086911470499b4a212660b02aa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:36.470734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1310/2593] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:30:36 2022] DELETE /identity/v3/users/8f4b4e086911470499b4a212660b02aa => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:36.475162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f0f07-a7db-48f7-a15f-63be7cc9b804 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:36.475448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f0f07-a7db-48f7-a15f-63be7cc9b804 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:36.475728 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f0f07-a7db-48f7-a15f-63be7cc9b804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:36.495352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-130f0f07-a7db-48f7-a15f-63be7cc9b804 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:36.496572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-130f0f07-a7db-48f7-a15f-63be7cc9b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:36.560524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1284/2594] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:30:36.570406 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:36.571857 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:36.629729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b510c60-17e4-4a83-936d-cf4eb3e032b2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:36.633597 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cce03-a934-4f97-8eae-7d93cc727cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:36.634628 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cce03-a934-4f97-8eae-7d93cc727cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:36.635118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cce03-a934-4f97-8eae-7d93cc727cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:36.635506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cce03-a934-4f97-8eae-7d93cc727cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:36.646967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cce03-a934-4f97-8eae-7d93cc727cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:36.648594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cce03-a934-4f97-8eae-7d93cc727cfd service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:36.658684 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1311/2595] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:37.069197 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f0f07-a7db-48f7-a15f-63be7cc9b804 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:37.070619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-130f0f07-a7db-48f7-a15f-63be7cc9b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:37.085092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f0f07-a7db-48f7-a15f-63be7cc9b804 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:37.090202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:37.091469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:37.092179 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:37.092473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] PATH_INFO: `/v3/projects/aca06d25bb334255b4ae79dc225cb326` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:37.093649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca06d25bb334255b4ae79dc225cb326)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:37.095616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:37.193037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca06d25bb334255b4ae79dc225cb326', 'request_id': 'req-f51b1469-b7ee-4746-8093-fcf4416e0d9e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.250055 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca06d25bb334255b4ae79dc225cb326', 'request_id': 'req-f51b1469-b7ee-4746-8093-fcf4416e0d9e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.255754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca06d25bb334255b4ae79dc225cb326', 'request_id': 'req-f51b1469-b7ee-4746-8093-fcf4416e0d9e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.267586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] The token cache is being invalidate because project aca06d25bb334255b4ae79dc225cb326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:30:37.267875 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.268496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.269978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1285/2596] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:30:37 2022] DELETE /identity/v3/projects/aca06d25bb334255b4ae79dc225cb326 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:37.273900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cce03-a934-4f97-8eae-7d93cc727cfd service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:37.284129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cce03-a934-4f97-8eae-7d93cc727cfd service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:37.291845 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:37.342137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:37.342681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:37.343066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:37.343272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] PATH_INFO: `/v3/users/7b6d19ed4a074a7b991ee26aac7cc891` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:37.344399 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6d19ed4a074a7b991ee26aac7cc891)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:37.346308 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:37.507822 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6d19ed4a074a7b991ee26aac7cc891'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.515976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6d19ed4a074a7b991ee26aac7cc891'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.527618 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6d19ed4a074a7b991ee26aac7cc891'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.528374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6d19ed4a074a7b991ee26aac7cc891'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.529031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-862bc7b9-10e1-40e0-99ba-d6d11827f94f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6d19ed4a074a7b991ee26aac7cc891'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:37.585671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1312/2597] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:30:37 2022] DELETE /identity/v3/users/7b6d19ed4a074a7b991ee26aac7cc891 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:37.977734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:37.979287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:37.994560 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b1469-b7ee-4746-8093-fcf4416e0d9e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:38.027039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:38.060444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:38.061142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.061477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.061749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] PATH_INFO: `/v3/projects/c1cbf892823c42eb844fda97cb3d6b9e` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.062972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1cbf892823c42eb844fda97cb3d6b9e)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:38.065186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:38.301567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1cbf892823c42eb844fda97cb3d6b9e', 'request_id': 'req-0c542281-7f3c-40a0-a31d-ba674ecb7deb'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:38.364557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1cbf892823c42eb844fda97cb3d6b9e', 'request_id': 'req-0c542281-7f3c-40a0-a31d-ba674ecb7deb'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:38.371041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1cbf892823c42eb844fda97cb3d6b9e', 'request_id': 'req-0c542281-7f3c-40a0-a31d-ba674ecb7deb'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:38.383129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] The token cache is being invalidate because project c1cbf892823c42eb844fda97cb3d6b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:30:38.383415 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:38.384040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0c542281-7f3c-40a0-a31d-ba674ecb7deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:38.385558 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1286/2598] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:30:37 2022] DELETE /identity/v3/projects/c1cbf892823c42eb844fda97cb3d6b9e => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:38.391554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e28220-505f-4ca3-ba7f-30538d99d2fb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.392019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e28220-505f-4ca3-ba7f-30538d99d2fb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.392131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e28220-505f-4ca3-ba7f-30538d99d2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.411855 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-74e28220-505f-4ca3-ba7f-30538d99d2fb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:38.413040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-74e28220-505f-4ca3-ba7f-30538d99d2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:38.472463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1313/2599] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:38.482186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f70ad-df22-479c-88c8-89b66d523109 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.482475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f70ad-df22-479c-88c8-89b66d523109 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.482748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f70ad-df22-479c-88c8-89b66d523109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.502063 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-230f70ad-df22-479c-88c8-89b66d523109 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:38.503332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-230f70ad-df22-479c-88c8-89b66d523109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:38.510971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1287/2600] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:38.515997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e28220-505f-4ca3-ba7f-30538d99d2fb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:38.525724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e28220-505f-4ca3-ba7f-30538d99d2fb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:38.530761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:38.531387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.531668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.531946 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.532631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:30:38.532918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:38.534502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:38.546799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1314/2601] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:30:38.551037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f70ad-df22-479c-88c8-89b66d523109 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:38.560279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f70ad-df22-479c-88c8-89b66d523109 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:38.564809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df08d-523e-4621-b35a-59389aef3710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:38.565395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df08d-523e-4621-b35a-59389aef3710 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.565667 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df08d-523e-4621-b35a-59389aef3710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.565932 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df08d-523e-4621-b35a-59389aef3710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.566534 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05df08d-523e-4621-b35a-59389aef3710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:38.572258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05df08d-523e-4621-b35a-59389aef3710 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:38.648683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1288/2602] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:38 2022] POST /identity/v3/projects => generated 399 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:38.652862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:38.663421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3106be-65d3-4cde-86d0-559247374dcf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:38.676033 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:38.719539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5eb35-6394-4de9-8bd1-a21323ecc872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:38.720140 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5eb35-6394-4de9-8bd1-a21323ecc872 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.720420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5eb35-6394-4de9-8bd1-a21323ecc872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.720690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5eb35-6394-4de9-8bd1-a21323ecc872 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.721437 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5eb35-6394-4de9-8bd1-a21323ecc872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:38.723356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5eb35-6394-4de9-8bd1-a21323ecc872 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:38.884824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1315/2603] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:38 2022] POST /identity/v3/users => generated 300 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:38.889840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df08d-523e-4621-b35a-59389aef3710 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:38.900329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df08d-523e-4621-b35a-59389aef3710 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:38.905315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f64a4-6219-4ed1-8a6d-a4a114da5112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:38.905904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f64a4-6219-4ed1-8a6d-a4a114da5112 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.906208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f64a4-6219-4ed1-8a6d-a4a114da5112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.906481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f64a4-6219-4ed1-8a6d-a4a114da5112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.907191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f64a4-6219-4ed1-8a6d-a4a114da5112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:38.908782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f64a4-6219-4ed1-8a6d-a4a114da5112 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:38.918490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1289/2604] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:38.922564 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5eb35-6394-4de9-8bd1-a21323ecc872 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:38.932404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5eb35-6394-4de9-8bd1-a21323ecc872 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:38.938208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c458e-23c2-4d78-b425-deba6b475840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:38.938792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c458e-23c2-4d78-b425-deba6b475840 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.939100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c458e-23c2-4d78-b425-deba6b475840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.939373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c458e-23c2-4d78-b425-deba6b475840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.940007 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c458e-23c2-4d78-b425-deba6b475840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:38.941566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c458e-23c2-4d78-b425-deba6b475840 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:38.952574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1316/2605] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:30:38.956692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f64a4-6219-4ed1-8a6d-a4a114da5112 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:38.966795 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f64a4-6219-4ed1-8a6d-a4a114da5112 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:38.971528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44abc31-1599-4aa1-8be4-1ae1855eea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:38.972164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44abc31-1599-4aa1-8be4-1ae1855eea13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:38.972440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44abc31-1599-4aa1-8be4-1ae1855eea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:38.972711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44abc31-1599-4aa1-8be4-1ae1855eea13 admin admin] PATH_INFO: `/v3/projects/aef6f521405c4e2b919c4bda68a09b57/users/4338a8b1d99d41f9a8d7620f8a0b6779/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:38.987096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44abc31-1599-4aa1-8be4-1ae1855eea13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef6f521405c4e2b919c4bda68a09b57, user_id=4338a8b1d99d41f9a8d7620f8a0b6779, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:38.989755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44abc31-1599-4aa1-8be4-1ae1855eea13 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:39.049146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1290/2606] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:38 2022] PUT /identity/v3/projects/aef6f521405c4e2b919c4bda68a09b57/users/4338a8b1d99d41f9a8d7620f8a0b6779/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:39.059497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adb5f-d431-404b-9258-e781cc968357 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.059785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adb5f-d431-404b-9258-e781cc968357 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.060060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adb5f-d431-404b-9258-e781cc968357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.087521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b29adb5f-d431-404b-9258-e781cc968357 None None] MFA Rules not processed for user `4338a8b1d99d41f9a8d7620f8a0b6779`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:39.088746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b29adb5f-d431-404b-9258-e781cc968357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:39.157255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1317/2607] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:39.161796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44abc31-1599-4aa1-8be4-1ae1855eea13 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:39.213135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:39.223563 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44abc31-1599-4aa1-8be4-1ae1855eea13 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:39.228213 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1827b-21e9-47fb-aec7-d23d959688a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:39.228814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1827b-21e9-47fb-aec7-d23d959688a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.229093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1827b-21e9-47fb-aec7-d23d959688a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.229367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1827b-21e9-47fb-aec7-d23d959688a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.229996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1827b-21e9-47fb-aec7-d23d959688a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:39.232072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1827b-21e9-47fb-aec7-d23d959688a9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:39.315222 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1291/2608] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:39 2022] POST /identity/v3/projects => generated 399 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:39.320786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29adb5f-d431-404b-9258-e781cc968357 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:39.331554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29adb5f-d431-404b-9258-e781cc968357 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:39.387412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e795bb5-06a5-4a25-ab20-b06ea4e59909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:39.388072 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e795bb5-06a5-4a25-ab20-b06ea4e59909 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.403518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e795bb5-06a5-4a25-ab20-b06ea4e59909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.403518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e795bb5-06a5-4a25-ab20-b06ea4e59909 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.403518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e795bb5-06a5-4a25-ab20-b06ea4e59909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:39.403518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e795bb5-06a5-4a25-ab20-b06ea4e59909 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:39.485132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1318/2609] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:39 2022] POST /identity/v3/users => generated 306 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:39.489081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1827b-21e9-47fb-aec7-d23d959688a9 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:39.499810 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1827b-21e9-47fb-aec7-d23d959688a9 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:39.504320 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cf7ff-b964-4fff-8ac2-111cc99ba073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:39.504906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cf7ff-b964-4fff-8ac2-111cc99ba073 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.505180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cf7ff-b964-4fff-8ac2-111cc99ba073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.505450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cf7ff-b964-4fff-8ac2-111cc99ba073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.506091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cf7ff-b964-4fff-8ac2-111cc99ba073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:39.507780 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cf7ff-b964-4fff-8ac2-111cc99ba073 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:39.518893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1292/2610] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:39.523140 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e795bb5-06a5-4a25-ab20-b06ea4e59909 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:39.533304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e795bb5-06a5-4a25-ab20-b06ea4e59909 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:39.538001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcc941-e31d-430d-b697-ea584e81fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:39.538646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcc941-e31d-430d-b697-ea584e81fb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.538931 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcc941-e31d-430d-b697-ea584e81fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.539241 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcc941-e31d-430d-b697-ea584e81fb81 admin admin] PATH_INFO: `/v3/projects/a0301fc9b87f469a97e2c74a216975cd/users/aed3122129214f7d9431b0934336caa1/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.553288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:39.557071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcc941-e31d-430d-b697-ea584e81fb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0301fc9b87f469a97e2c74a216975cd, user_id=aed3122129214f7d9431b0934336caa1, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:39.560456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcc941-e31d-430d-b697-ea584e81fb81 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:39.671749 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1319/2611] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:39 2022] PUT /identity/v3/projects/a0301fc9b87f469a97e2c74a216975cd/users/aed3122129214f7d9431b0934336caa1/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:30:39.675791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cf7ff-b964-4fff-8ac2-111cc99ba073 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:39.685833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cf7ff-b964-4fff-8ac2-111cc99ba073 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:39.742533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbca7a-a0c3-407a-b0e3-f08c141b4359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:39.743167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbca7a-a0c3-407a-b0e3-f08c141b4359 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.743463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbca7a-a0c3-407a-b0e3-f08c141b4359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.743898 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbca7a-a0c3-407a-b0e3-f08c141b4359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.744453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbca7a-a0c3-407a-b0e3-f08c141b4359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:39.746075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbca7a-a0c3-407a-b0e3-f08c141b4359 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:39.756110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1293/2612] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:39.760501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcc941-e31d-430d-b697-ea584e81fb81 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:39.771131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcc941-e31d-430d-b697-ea584e81fb81 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:39.776234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:39.776873 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.777158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.777547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] PATH_INFO: `/v3/projects/a0301fc9b87f469a97e2c74a216975cd/users/aed3122129214f7d9431b0934336caa1/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.780104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0301fc9b87f469a97e2c74a216975cd, user_id=aed3122129214f7d9431b0934336caa1, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:39.782707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:39.793096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1320/2613] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:39 2022] PUT /identity/v3/projects/a0301fc9b87f469a97e2c74a216975cd/users/aed3122129214f7d9431b0934336caa1/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:39.803911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e35c61-2771-40b7-af75-9325deea2705 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.804195 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e35c61-2771-40b7-af75-9325deea2705 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.804468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e35c61-2771-40b7-af75-9325deea2705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.832059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-40e35c61-2771-40b7-af75-9325deea2705 None None] MFA Rules not processed for user `aed3122129214f7d9431b0934336caa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:39.833269 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-40e35c61-2771-40b7-af75-9325deea2705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:39.900003 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1294/2614] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:39 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:39.911750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:39.913439 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:39.969893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568710f0-423f-4d7a-a9b1-4b69bc72b1e4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:39.973447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736e124-d8a9-417d-a9a5-ae36054f6044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:39.974037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736e124-d8a9-417d-a9a5-ae36054f6044 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:39.974314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736e124-d8a9-417d-a9a5-ae36054f6044 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:39.974586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736e124-d8a9-417d-a9a5-ae36054f6044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:39.984174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736e124-d8a9-417d-a9a5-ae36054f6044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:39.985578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736e124-d8a9-417d-a9a5-ae36054f6044 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:39.994848 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1321/2615] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:40.196414 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e35c61-2771-40b7-af75-9325deea2705 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:40.206744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e35c61-2771-40b7-af75-9325deea2705 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:40.210258 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b056c5-55ac-414d-86e3-efe95d2bf1e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:40.210852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b056c5-55ac-414d-86e3-efe95d2bf1e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:40.211168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b056c5-55ac-414d-86e3-efe95d2bf1e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:40.211446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b056c5-55ac-414d-86e3-efe95d2bf1e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:40.257226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:40.269655 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b056c5-55ac-414d-86e3-efe95d2bf1e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:40.271120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b056c5-55ac-414d-86e3-efe95d2bf1e4 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:40.280985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1295/2616] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:40 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:45.170616 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736e124-d8a9-417d-a9a5-ae36054f6044 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:45.233571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736e124-d8a9-417d-a9a5-ae36054f6044 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:45.238082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:45.238696 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:45.238976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:45.239288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] PATH_INFO: `/v3/users/4338a8b1d99d41f9a8d7620f8a0b6779` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:45.240423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4338a8b1d99d41f9a8d7620f8a0b6779)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:45.242336 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:45.409054 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4338a8b1d99d41f9a8d7620f8a0b6779'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:45.415752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4338a8b1d99d41f9a8d7620f8a0b6779'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:45.428022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4338a8b1d99d41f9a8d7620f8a0b6779'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:45.428609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4338a8b1d99d41f9a8d7620f8a0b6779'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:45.429224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4338a8b1d99d41f9a8d7620f8a0b6779'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:45.491388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:45.495468 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1322/2617] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:30:45 2022] DELETE /identity/v3/users/4338a8b1d99d41f9a8d7620f8a0b6779 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:45.500855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390ec08-2b72-461c-887d-f921397ef3ae None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:45.509243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390ec08-2b72-461c-887d-f921397ef3ae None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:45.509243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390ec08-2b72-461c-887d-f921397ef3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:45.519938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6390ec08-2b72-461c-887d-f921397ef3ae None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:45.521135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6390ec08-2b72-461c-887d-f921397ef3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:45.577975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1296/2618] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:45.587629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:45.589206 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:45.646656 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c9bc0-708d-499b-9529-93ab419ce580 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:45.650209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28443779-c786-4841-b758-e7beb8c0ca4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:45.650826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28443779-c786-4841-b758-e7beb8c0ca4f service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:45.651128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28443779-c786-4841-b758-e7beb8c0ca4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:45.651400 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28443779-c786-4841-b758-e7beb8c0ca4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:45.660765 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28443779-c786-4841-b758-e7beb8c0ca4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:45.662163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28443779-c786-4841-b758-e7beb8c0ca4f service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:45.672226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1323/2619] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:46.063019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390ec08-2b72-461c-887d-f921397ef3ae None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:46.067603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6390ec08-2b72-461c-887d-f921397ef3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:46.076958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390ec08-2b72-461c-887d-f921397ef3ae None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:46.081794 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:46.084826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:46.084826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:46.084826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] PATH_INFO: `/v3/projects/aef6f521405c4e2b919c4bda68a09b57` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:46.084826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef6f521405c4e2b919c4bda68a09b57)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:46.085972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:46.187507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef6f521405c4e2b919c4bda68a09b57', 'request_id': 'req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.270172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef6f521405c4e2b919c4bda68a09b57', 'request_id': 'req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.275827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef6f521405c4e2b919c4bda68a09b57', 'request_id': 'req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.287262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] The token cache is being invalidate because project aef6f521405c4e2b919c4bda68a09b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:30:46.287551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.288171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.289629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1297/2620] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:30:46 2022] DELETE /identity/v3/projects/aef6f521405c4e2b919c4bda68a09b57 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:46.294041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28443779-c786-4841-b758-e7beb8c0ca4f service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:46.303818 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28443779-c786-4841-b758-e7beb8c0ca4f service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:46.359122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:46.359745 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:46.360030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:46.360307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] PATH_INFO: `/v3/users/aed3122129214f7d9431b0934336caa1` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:46.361451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed3122129214f7d9431b0934336caa1)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:46.363394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:46.546358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed3122129214f7d9431b0934336caa1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.553286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed3122129214f7d9431b0934336caa1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.565757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed3122129214f7d9431b0934336caa1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.566526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed3122129214f7d9431b0934336caa1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.567170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-51c58920-b717-4f07-b7f7-65d8dabf2578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed3122129214f7d9431b0934336caa1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:46.642864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1324/2621] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:30:46 2022] DELETE /identity/v3/users/aed3122129214f7d9431b0934336caa1 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:47.079961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:47.081481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:47.095663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2b2fa-ec6f-402d-bb15-5ec837e2351d admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:47.157813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:47.158436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.158710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.158979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] PATH_INFO: `/v3/projects/a0301fc9b87f469a97e2c74a216975cd` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.160112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0301fc9b87f469a97e2c74a216975cd)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:47.161980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:47.173270 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:47.260060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0301fc9b87f469a97e2c74a216975cd', 'request_id': 'req-4aad4563-4da6-4604-ba6d-e9a42486441d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:47.318151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0301fc9b87f469a97e2c74a216975cd', 'request_id': 'req-4aad4563-4da6-4604-ba6d-e9a42486441d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:47.323717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0301fc9b87f469a97e2c74a216975cd', 'request_id': 'req-4aad4563-4da6-4604-ba6d-e9a42486441d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:47.335216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] The token cache is being invalidate because project a0301fc9b87f469a97e2c74a216975cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:30:47.335503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:47.336106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4aad4563-4da6-4604-ba6d-e9a42486441d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:47.337666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1298/2622] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:30:47 2022] DELETE /identity/v3/projects/a0301fc9b87f469a97e2c74a216975cd => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:47.342723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c8e72-6305-4a84-a4cd-543d21477ccd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.343236 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c8e72-6305-4a84-a4cd-543d21477ccd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.343356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c8e72-6305-4a84-a4cd-543d21477ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.363372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7a6c8e72-6305-4a84-a4cd-543d21477ccd None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:47.364566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7a6c8e72-6305-4a84-a4cd-543d21477ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:47.381763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:47.423480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1325/2623] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:30:47.432199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9116f2-0c6c-42bf-a4ff-9fdcdff9810a None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.436720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9116f2-0c6c-42bf-a4ff-9fdcdff9810a None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.436720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9116f2-0c6c-42bf-a4ff-9fdcdff9810a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.453167 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-8e9116f2-0c6c-42bf-a4ff-9fdcdff9810a None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:47.454350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8e9116f2-0c6c-42bf-a4ff-9fdcdff9810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:47.462116 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1299/2624] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:47.466470 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c8e72-6305-4a84-a4cd-543d21477ccd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:47.475799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c8e72-6305-4a84-a4cd-543d21477ccd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:47.480389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:47.480980 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.481260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.481542 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.482174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:30:47.482480 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:47.484163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:47.496452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1326/2625] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 12:30:47.501233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9116f2-0c6c-42bf-a4ff-9fdcdff9810a None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:47.510243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9116f2-0c6c-42bf-a4ff-9fdcdff9810a None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:47.515907 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe02e53-1120-4d1f-ad66-d5a14ee6e74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:47.516491 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02e53-1120-4d1f-ad66-d5a14ee6e74e admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.516766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02e53-1120-4d1f-ad66-d5a14ee6e74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.517034 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02e53-1120-4d1f-ad66-d5a14ee6e74e admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.517641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe02e53-1120-4d1f-ad66-d5a14ee6e74e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:47.519556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe02e53-1120-4d1f-ad66-d5a14ee6e74e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:47.599365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1300/2626] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:47 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:47.603403 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:47.613901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86804898-237e-4ff3-9fd6-8155b53b8a8a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:47.669352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95fb1b-4759-4a80-a720-3a1ea2562d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:47.669938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95fb1b-4759-4a80-a720-3a1ea2562d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.670212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95fb1b-4759-4a80-a720-3a1ea2562d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.670476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95fb1b-4759-4a80-a720-3a1ea2562d59 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.671115 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95fb1b-4759-4a80-a720-3a1ea2562d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:47.673000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95fb1b-4759-4a80-a720-3a1ea2562d59 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:47.775881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1327/2627] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:47 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:47.780165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe02e53-1120-4d1f-ad66-d5a14ee6e74e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:47.790273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe02e53-1120-4d1f-ad66-d5a14ee6e74e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:47.795617 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec6fa6-3edc-4025-abcc-a7fac9f11ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:47.796201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec6fa6-3edc-4025-abcc-a7fac9f11ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.796475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec6fa6-3edc-4025-abcc-a7fac9f11ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.796742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec6fa6-3edc-4025-abcc-a7fac9f11ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.797472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec6fa6-3edc-4025-abcc-a7fac9f11ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:47.799062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec6fa6-3edc-4025-abcc-a7fac9f11ce4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:47.808964 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1301/2628] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:47.813788 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95fb1b-4759-4a80-a720-3a1ea2562d59 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:47.823903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95fb1b-4759-4a80-a720-3a1ea2562d59 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:47.828646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c7961-64fa-4a00-b5fb-589a8cbbe5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:47.829240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7961-64fa-4a00-b5fb-589a8cbbe5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.829521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7961-64fa-4a00-b5fb-589a8cbbe5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.829795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7961-64fa-4a00-b5fb-589a8cbbe5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.830472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c7961-64fa-4a00-b5fb-589a8cbbe5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:47.832083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c7961-64fa-4a00-b5fb-589a8cbbe5d5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:47.843021 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1328/2629] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:30:47.847124 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec6fa6-3edc-4025-abcc-a7fac9f11ce4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:47.857588 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec6fa6-3edc-4025-abcc-a7fac9f11ce4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:47.862106 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7fbc-5bdc-4aa8-965f-238e97b67476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:47.862733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7fbc-5bdc-4aa8-965f-238e97b67476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.863030 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7fbc-5bdc-4aa8-965f-238e97b67476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.863310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7fbc-5bdc-4aa8-965f-238e97b67476 admin admin] PATH_INFO: `/v3/projects/a810b25198ee4323b49eb81aa890fe72/users/f7096b7115644e7aad96a6ecfafaf712/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.879046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf7fbc-5bdc-4aa8-965f-238e97b67476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a810b25198ee4323b49eb81aa890fe72, user_id=f7096b7115644e7aad96a6ecfafaf712, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:47.881714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf7fbc-5bdc-4aa8-965f-238e97b67476 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:47.946525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1302/2630] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:47 2022] PUT /identity/v3/projects/a810b25198ee4323b49eb81aa890fe72/users/f7096b7115644e7aad96a6ecfafaf712/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:47.958954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b004ca0-a76f-431d-92be-b8d283bed804 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:47.959271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b004ca0-a76f-431d-92be-b8d283bed804 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:47.959572 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b004ca0-a76f-431d-92be-b8d283bed804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:47.989521 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2b004ca0-a76f-431d-92be-b8d283bed804 None None] MFA Rules not processed for user `f7096b7115644e7aad96a6ecfafaf712`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:47.990709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2b004ca0-a76f-431d-92be-b8d283bed804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:48.055938 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1329/2631] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:48.060848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7fbc-5bdc-4aa8-965f-238e97b67476 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:48.121068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7fbc-5bdc-4aa8-965f-238e97b67476 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:48.126251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e5c2-26fc-452c-9bb3-ece67266b0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:48.126853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e5c2-26fc-452c-9bb3-ece67266b0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:48.127159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e5c2-26fc-452c-9bb3-ece67266b0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:48.127427 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e5c2-26fc-452c-9bb3-ece67266b0b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:48.128037 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e5c2-26fc-452c-9bb3-ece67266b0b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:48.129914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e5c2-26fc-452c-9bb3-ece67266b0b5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:48.234361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1303/2632] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:48 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:48.238673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b004ca0-a76f-431d-92be-b8d283bed804 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:48.249780 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b004ca0-a76f-431d-92be-b8d283bed804 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:48.261781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:48.306040 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf496c4-8463-4090-85b2-b4e214660d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:48.306641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf496c4-8463-4090-85b2-b4e214660d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:48.306918 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf496c4-8463-4090-85b2-b4e214660d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:48.307217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf496c4-8463-4090-85b2-b4e214660d75 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:48.307842 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf496c4-8463-4090-85b2-b4e214660d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:48.310366 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf496c4-8463-4090-85b2-b4e214660d75 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:48.398915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1330/2633] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:48 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:48.403186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e5c2-26fc-452c-9bb3-ece67266b0b5 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:48.413711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e5c2-26fc-452c-9bb3-ece67266b0b5 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:48.419048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bd88f-821a-4114-8a9f-d9c81f4563cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:48.419736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd88f-821a-4114-8a9f-d9c81f4563cd admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:48.420024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd88f-821a-4114-8a9f-d9c81f4563cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:48.420298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd88f-821a-4114-8a9f-d9c81f4563cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:48.420969 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bd88f-821a-4114-8a9f-d9c81f4563cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:48.422602 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bd88f-821a-4114-8a9f-d9c81f4563cd admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:48.433178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1304/2634] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:48.438245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf496c4-8463-4090-85b2-b4e214660d75 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:48.449046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf496c4-8463-4090-85b2-b4e214660d75 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:48.454210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15add98d-6223-4ad5-9bb3-d18020e37fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:48.456499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15add98d-6223-4ad5-9bb3-d18020e37fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:48.456499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15add98d-6223-4ad5-9bb3-d18020e37fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:48.456499 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15add98d-6223-4ad5-9bb3-d18020e37fd0 admin admin] PATH_INFO: `/v3/projects/1559b668bb3c42028e102447544e78d5/users/1f63fbab36994010bc1fab114539800f/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:48.470690 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15add98d-6223-4ad5-9bb3-d18020e37fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1559b668bb3c42028e102447544e78d5, user_id=1f63fbab36994010bc1fab114539800f, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:48.473343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15add98d-6223-4ad5-9bb3-d18020e37fd0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:48.550630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1331/2635] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:48 2022] PUT /identity/v3/projects/1559b668bb3c42028e102447544e78d5/users/1f63fbab36994010bc1fab114539800f/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:48.554535 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bd88f-821a-4114-8a9f-d9c81f4563cd admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:48.565198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bd88f-821a-4114-8a9f-d9c81f4563cd admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:48.572683 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:48.619806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71452ea8-2598-4b9c-b98b-488f0f3a147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:48.620400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452ea8-2598-4b9c-b98b-488f0f3a147e admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:48.620705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452ea8-2598-4b9c-b98b-488f0f3a147e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:48.620980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452ea8-2598-4b9c-b98b-488f0f3a147e admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:48.622399 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71452ea8-2598-4b9c-b98b-488f0f3a147e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:48.624032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71452ea8-2598-4b9c-b98b-488f0f3a147e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:48.633420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1305/2636] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:48.637778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15add98d-6223-4ad5-9bb3-d18020e37fd0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:48.648238 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15add98d-6223-4ad5-9bb3-d18020e37fd0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:48.654006 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:48.654141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:48.654433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:48.654710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] PATH_INFO: `/v3/projects/1559b668bb3c42028e102447544e78d5/users/1f63fbab36994010bc1fab114539800f/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:48.656647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1559b668bb3c42028e102447544e78d5, user_id=1f63fbab36994010bc1fab114539800f, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:48.659273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:48.669979 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1332/2637] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:48 2022] PUT /identity/v3/projects/1559b668bb3c42028e102447544e78d5/users/1f63fbab36994010bc1fab114539800f/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:48.680843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49102de-8634-4b6b-a801-5c5cdc3d5244 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:48.681130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49102de-8634-4b6b-a801-5c5cdc3d5244 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:48.681406 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49102de-8634-4b6b-a801-5c5cdc3d5244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:48.708638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d49102de-8634-4b6b-a801-5c5cdc3d5244 None None] MFA Rules not processed for user `1f63fbab36994010bc1fab114539800f`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:48.709852 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d49102de-8634-4b6b-a801-5c5cdc3d5244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:48.778152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1306/2638] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:48.789297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:48.791056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:48.849638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bc6e0-d159-4a3a-877a-e094eac5aee9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:48.853525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d643eb-bbb9-4caf-83b7-507b6f23c1c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:48.854122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d643eb-bbb9-4caf-83b7-507b6f23c1c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:48.854456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d643eb-bbb9-4caf-83b7-507b6f23c1c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:48.854728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d643eb-bbb9-4caf-83b7-507b6f23c1c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:48.863804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d643eb-bbb9-4caf-83b7-507b6f23c1c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:48.869140 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d643eb-bbb9-4caf-83b7-507b6f23c1c7 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:48.875503 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1333/2639] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:49.089490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49102de-8634-4b6b-a801-5c5cdc3d5244 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:49.099196 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49102de-8634-4b6b-a801-5c5cdc3d5244 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:49.102669 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ec738-7295-45b9-abf1-8c1e9f55df3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:49.103372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec738-7295-45b9-abf1-8c1e9f55df3e service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:49.103652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec738-7295-45b9-abf1-8c1e9f55df3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:49.103920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ec738-7295-45b9-abf1-8c1e9f55df3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:49.161202 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ec738-7295-45b9-abf1-8c1e9f55df3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:49.162648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ec738-7295-45b9-abf1-8c1e9f55df3e service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:49.172263 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1307/2640] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:55.376015 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d643eb-bbb9-4caf-83b7-507b6f23c1c7 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:55.438510 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d643eb-bbb9-4caf-83b7-507b6f23c1c7 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:55.443185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:55.443836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:55.444790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:55.445221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] PATH_INFO: `/v3/users/f7096b7115644e7aad96a6ecfafaf712` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:55.446372 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7096b7115644e7aad96a6ecfafaf712)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:55.448298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:55.462199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:55.634809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7096b7115644e7aad96a6ecfafaf712'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:55.642558 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7096b7115644e7aad96a6ecfafaf712'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:55.654126 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7096b7115644e7aad96a6ecfafaf712'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:55.654831 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7096b7115644e7aad96a6ecfafaf712'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:55.655456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7096b7115644e7aad96a6ecfafaf712'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:55.718819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1334/2641] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:30:55 2022] DELETE /identity/v3/users/f7096b7115644e7aad96a6ecfafaf712 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:55.723565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44627c-55e8-4d48-ace3-e64f9771acfb None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:55.724314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44627c-55e8-4d48-ace3-e64f9771acfb None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:55.724314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44627c-55e8-4d48-ace3-e64f9771acfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:55.744260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-9d44627c-55e8-4d48-ace3-e64f9771acfb None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:55.745528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9d44627c-55e8-4d48-ace3-e64f9771acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:55.773857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=100 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:55.807164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1308/2642] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:55.817585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:55.818976 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:55.877671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575c17e-1571-45c4-9801-bbabbec6d1f1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:55.881232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e8d85-8046-4e86-b235-c964b98c704a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:55.881826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e8d85-8046-4e86-b235-c964b98c704a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:55.882102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e8d85-8046-4e86-b235-c964b98c704a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:55.882371 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e8d85-8046-4e86-b235-c964b98c704a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:55.891556 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e8d85-8046-4e86-b235-c964b98c704a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:55.892909 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e8d85-8046-4e86-b235-c964b98c704a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:55.902296 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1335/2643] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:30:56.291323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44627c-55e8-4d48-ace3-e64f9771acfb None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:56.293424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9d44627c-55e8-4d48-ace3-e64f9771acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:56.304500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44627c-55e8-4d48-ace3-e64f9771acfb None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:56.309802 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:56.310433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:56.310711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:56.310985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] PATH_INFO: `/v3/projects/a810b25198ee4323b49eb81aa890fe72` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:56.312266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a810b25198ee4323b49eb81aa890fe72)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:56.314157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:56.420528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a810b25198ee4323b49eb81aa890fe72', 'request_id': 'req-796fc0ed-9966-4c56-962b-5a481b663a64'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.495453 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a810b25198ee4323b49eb81aa890fe72', 'request_id': 'req-796fc0ed-9966-4c56-962b-5a481b663a64'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.501813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a810b25198ee4323b49eb81aa890fe72', 'request_id': 'req-796fc0ed-9966-4c56-962b-5a481b663a64'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.514004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] The token cache is being invalidate because project a810b25198ee4323b49eb81aa890fe72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:30:56.514297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.514926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.516464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1309/2644] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:30:56 2022] DELETE /identity/v3/projects/a810b25198ee4323b49eb81aa890fe72 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:56.520471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e8d85-8046-4e86-b235-c964b98c704a service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:56.530411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e8d85-8046-4e86-b235-c964b98c704a service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:56.586413 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:56.587058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:56.587339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:56.587610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] PATH_INFO: `/v3/users/1f63fbab36994010bc1fab114539800f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:56.588709 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f63fbab36994010bc1fab114539800f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:56.590720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:56.761202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:56.769638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f63fbab36994010bc1fab114539800f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.776434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f63fbab36994010bc1fab114539800f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.789543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f63fbab36994010bc1fab114539800f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.790221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f63fbab36994010bc1fab114539800f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.790824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9323f760-7faf-4e6e-b877-45e669cbadb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f63fbab36994010bc1fab114539800f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:56.895198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1336/2645] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:30:56 2022] DELETE /identity/v3/users/1f63fbab36994010bc1fab114539800f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:57.314155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:57.315718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:57.330684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc0ed-9966-4c56-962b-5a481b663a64 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:57.389015 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:57.389664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:57.389944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:57.390219 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] PATH_INFO: `/v3/projects/1559b668bb3c42028e102447544e78d5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:57.391420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] RBAC: Authorizing `identity:delete_project(project_id=1559b668bb3c42028e102447544e78d5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:57.393336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:57.498821 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1559b668bb3c42028e102447544e78d5', 'request_id': 'req-869c6d01-3155-4f3f-b388-0f9f79ba78de'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:57.567711 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1559b668bb3c42028e102447544e78d5', 'request_id': 'req-869c6d01-3155-4f3f-b388-0f9f79ba78de'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:57.569253 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:57.574135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1559b668bb3c42028e102447544e78d5', 'request_id': 'req-869c6d01-3155-4f3f-b388-0f9f79ba78de'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:57.585793 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] The token cache is being invalidate because project 1559b668bb3c42028e102447544e78d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:30:57.586020 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:57.586646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-869c6d01-3155-4f3f-b388-0f9f79ba78de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:30:57.588165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1310/2646] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:30:57 2022] DELETE /identity/v3/projects/1559b668bb3c42028e102447544e78d5 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:57.593795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18579a1-d020-44b7-9139-07bb922e8c20 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:57.594344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18579a1-d020-44b7-9139-07bb922e8c20 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:57.594344 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18579a1-d020-44b7-9139-07bb922e8c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:57.614246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d18579a1-d020-44b7-9139-07bb922e8c20 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:57.615484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d18579a1-d020-44b7-9139-07bb922e8c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:57.678222 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1337/2647] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:57.687902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d585-17f0-40c3-aed8-52e9f911830c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:57.688186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d585-17f0-40c3-aed8-52e9f911830c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:57.697693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d585-17f0-40c3-aed8-52e9f911830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:57.708159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-81b8d585-17f0-40c3-aed8-52e9f911830c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:57.709439 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-81b8d585-17f0-40c3-aed8-52e9f911830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:57.717955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1311/2648] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:57.722272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18579a1-d020-44b7-9139-07bb922e8c20 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:57.732186 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18579a1-d020-44b7-9139-07bb922e8c20 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:57.736864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:57.737463 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:57.737743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:57.738017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:57.738657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:30:57.738957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:57.740576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:57.754002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1338/2649] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:30:57.758547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8d585-17f0-40c3-aed8-52e9f911830c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:57.768820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8d585-17f0-40c3-aed8-52e9f911830c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:57.773608 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86508760-977c-4c0e-9c4f-5fd90bbb7b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:57.774228 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86508760-977c-4c0e-9c4f-5fd90bbb7b14 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:57.774512 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86508760-977c-4c0e-9c4f-5fd90bbb7b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:57.774804 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86508760-977c-4c0e-9c4f-5fd90bbb7b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:57.775468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86508760-977c-4c0e-9c4f-5fd90bbb7b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:57.777505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86508760-977c-4c0e-9c4f-5fd90bbb7b14 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:57.866215 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1312/2650] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:57 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:57.870129 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:57.881750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3f740-8275-4874-b522-1d92bcbcfd62 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:57.942061 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376f2e5-f59a-45f5-9923-2ff4717543e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:57.942663 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376f2e5-f59a-45f5-9923-2ff4717543e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:57.942946 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376f2e5-f59a-45f5-9923-2ff4717543e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:57.943259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376f2e5-f59a-45f5-9923-2ff4717543e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:57.944011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376f2e5-f59a-45f5-9923-2ff4717543e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:57.945904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376f2e5-f59a-45f5-9923-2ff4717543e3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:58.016230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1339/2651] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:57 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:58.020332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86508760-977c-4c0e-9c4f-5fd90bbb7b14 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:58.030855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86508760-977c-4c0e-9c4f-5fd90bbb7b14 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:58.035495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda6fc2-b34b-4711-9239-bcb123b1e6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:58.036079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda6fc2-b34b-4711-9239-bcb123b1e6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:58.036354 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda6fc2-b34b-4711-9239-bcb123b1e6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:58.036710 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda6fc2-b34b-4711-9239-bcb123b1e6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:58.037377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda6fc2-b34b-4711-9239-bcb123b1e6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:58.038945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda6fc2-b34b-4711-9239-bcb123b1e6c1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:58.049363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1313/2652] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:30:58.053341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376f2e5-f59a-45f5-9923-2ff4717543e3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:58.064217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376f2e5-f59a-45f5-9923-2ff4717543e3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:58.068722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e435c2-f3fb-4536-b8bf-9170c7d805ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:58.069309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e435c2-f3fb-4536-b8bf-9170c7d805ec admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:58.069590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e435c2-f3fb-4536-b8bf-9170c7d805ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:58.069863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e435c2-f3fb-4536-b8bf-9170c7d805ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:58.070526 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e435c2-f3fb-4536-b8bf-9170c7d805ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:58.072128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e435c2-f3fb-4536-b8bf-9170c7d805ec admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:58.082586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1340/2653] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:30:58.086549 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda6fc2-b34b-4711-9239-bcb123b1e6c1 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:58.096551 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda6fc2-b34b-4711-9239-bcb123b1e6c1 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:58.101019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:58.101642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:58.101916 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:58.102187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] PATH_INFO: `/v3/projects/5b41a4f688ee478fb269f79ac83266d2/users/ad7686dcd7bb4b10ab6ff2f415613eb0/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:58.117268 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b41a4f688ee478fb269f79ac83266d2, user_id=ad7686dcd7bb4b10ab6ff2f415613eb0, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:58.119934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:58.181341 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1314/2654] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:30:58 2022] PUT /identity/v3/projects/5b41a4f688ee478fb269f79ac83266d2/users/ad7686dcd7bb4b10ab6ff2f415613eb0/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:30:58.191036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afbec4-4141-487d-8e3c-dd23fdff8b8e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:58.191314 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afbec4-4141-487d-8e3c-dd23fdff8b8e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:58.191588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afbec4-4141-487d-8e3c-dd23fdff8b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:58.210606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d9afbec4-4141-487d-8e3c-dd23fdff8b8e None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:58.211829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d9afbec4-4141-487d-8e3c-dd23fdff8b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:58.224297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:30:58.271256 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1341/2655] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:58.280952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:58.283723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:58.338787 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146bb7-1a4e-4f94-995c-c291f1d4f4ce admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:58.342463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56670e33-170c-4658-85df-9a44f5da9526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:58.343085 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56670e33-170c-4658-85df-9a44f5da9526 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:58.343364 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56670e33-170c-4658-85df-9a44f5da9526 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:58.343629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56670e33-170c-4658-85df-9a44f5da9526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:58.352217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56670e33-170c-4658-85df-9a44f5da9526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:58.353627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56670e33-170c-4658-85df-9a44f5da9526 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:58.363201 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1315/2656] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:30:59.604895 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7f1f-746e-41de-b343-0e15caba9991 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:59.605431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7f1f-746e-41de-b343-0e15caba9991 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:59.605431 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7f1f-746e-41de-b343-0e15caba9991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:59.633454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-198b7f1f-746e-41de-b343-0e15caba9991 None None] MFA Rules not processed for user `ad7686dcd7bb4b10ab6ff2f415613eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:30:59.634666 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-198b7f1f-746e-41de-b343-0e15caba9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:30:59.702846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1342/2657] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:30:59.707872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56670e33-170c-4658-85df-9a44f5da9526 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:59.717876 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56670e33-170c-4658-85df-9a44f5da9526 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:59.722440 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670420a-5c9a-410f-8231-73fd6ec08e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:59.723098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670420a-5c9a-410f-8231-73fd6ec08e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:59.723376 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670420a-5c9a-410f-8231-73fd6ec08e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:59.723648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670420a-5c9a-410f-8231-73fd6ec08e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:59.724273 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670420a-5c9a-410f-8231-73fd6ec08e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:59.726188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670420a-5c9a-410f-8231-73fd6ec08e75 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:59.809321 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1316/2658] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:30:59 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:30:59.813385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b7f1f-746e-41de-b343-0e15caba9991 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:59.824032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b7f1f-746e-41de-b343-0e15caba9991 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:59.882068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d106f7-3abf-4554-8574-b281cb26e5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:59.882689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d106f7-3abf-4554-8574-b281cb26e5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:59.882966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d106f7-3abf-4554-8574-b281cb26e5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:59.883278 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d106f7-3abf-4554-8574-b281cb26e5a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:59.883887 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d106f7-3abf-4554-8574-b281cb26e5a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:59.885770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d106f7-3abf-4554-8574-b281cb26e5a2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:30:59.975294 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1343/2659] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:30:59 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:30:59.979301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670420a-5c9a-410f-8231-73fd6ec08e75 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:30:59.989618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670420a-5c9a-410f-8231-73fd6ec08e75 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:30:59.994145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5a3e3-c5a0-488e-93fd-aa2020724bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:30:59.994762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5a3e3-c5a0-488e-93fd-aa2020724bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:30:59.995072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5a3e3-c5a0-488e-93fd-aa2020724bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:30:59.995345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5a3e3-c5a0-488e-93fd-aa2020724bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:30:59.995979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc5a3e3-c5a0-488e-93fd-aa2020724bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:30:59.998054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc5a3e3-c5a0-488e-93fd-aa2020724bfe admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:00.009386 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1317/2660] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:31:00.014647 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d106f7-3abf-4554-8574-b281cb26e5a2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:00.024639 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d106f7-3abf-4554-8574-b281cb26e5a2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:00.029274 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90505135-488c-4011-9d8f-def1636bc87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:00.029913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90505135-488c-4011-9d8f-def1636bc87c admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:00.030191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90505135-488c-4011-9d8f-def1636bc87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:00.030471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90505135-488c-4011-9d8f-def1636bc87c admin admin] PATH_INFO: `/v3/projects/4bcd36e2524c4adaaa3b20ba79ba5e61/users/065a58a3cfbc4898a1896dc93530cf1d/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:00.044910 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90505135-488c-4011-9d8f-def1636bc87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bcd36e2524c4adaaa3b20ba79ba5e61, user_id=065a58a3cfbc4898a1896dc93530cf1d, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:00.047559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90505135-488c-4011-9d8f-def1636bc87c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:00.116763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1344/2661] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:31:00 2022] PUT /identity/v3/projects/4bcd36e2524c4adaaa3b20ba79ba5e61/users/065a58a3cfbc4898a1896dc93530cf1d/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:31:00.120887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5a3e3-c5a0-488e-93fd-aa2020724bfe admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:00.131445 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5a3e3-c5a0-488e-93fd-aa2020724bfe admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:00.177509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:00.187603 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf3e9f-c440-4bc2-b6b0-f0507c858be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:00.188206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf3e9f-c440-4bc2-b6b0-f0507c858be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:00.188733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf3e9f-c440-4bc2-b6b0-f0507c858be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:00.188733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf3e9f-c440-4bc2-b6b0-f0507c858be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:00.189367 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf3e9f-c440-4bc2-b6b0-f0507c858be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:00.190955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf3e9f-c440-4bc2-b6b0-f0507c858be9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:00.201457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1318/2662] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:31:00.205910 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90505135-488c-4011-9d8f-def1636bc87c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:00.216481 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90505135-488c-4011-9d8f-def1636bc87c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:00.221084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:00.221729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:00.222009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:00.222285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] PATH_INFO: `/v3/projects/4bcd36e2524c4adaaa3b20ba79ba5e61/users/065a58a3cfbc4898a1896dc93530cf1d/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:00.224303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bcd36e2524c4adaaa3b20ba79ba5e61, user_id=065a58a3cfbc4898a1896dc93530cf1d, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:00.226901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:00.237223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1345/2663] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:31:00 2022] PUT /identity/v3/projects/4bcd36e2524c4adaaa3b20ba79ba5e61/users/065a58a3cfbc4898a1896dc93530cf1d/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:01.548880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13b392-ffa4-414e-a814-340fdc346df6 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:01.549483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13b392-ffa4-414e-a814-340fdc346df6 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:01.549483 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13b392-ffa4-414e-a814-340fdc346df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:01.576760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-af13b392-ffa4-414e-a814-340fdc346df6 None None] MFA Rules not processed for user `065a58a3cfbc4898a1896dc93530cf1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:01.577995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-af13b392-ffa4-414e-a814-340fdc346df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:01.644975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1319/2664] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:01.656635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:01.658332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:01.663726 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:01.717383 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc569b18-0e37-45db-a9ff-322e60488590 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:01.720912 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:01.721531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:01.721806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:01.722080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:01.780515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:01.781945 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:01.790683 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1346/2665] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:31:03.462041 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13b392-ffa4-414e-a814-340fdc346df6 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:03.463857 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-af13b392-ffa4-414e-a814-340fdc346df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:03.476021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13b392-ffa4-414e-a814-340fdc346df6 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:03.480171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93dcfa-9bea-411a-afe3-d2fdfcfb68f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:03.480941 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93dcfa-9bea-411a-afe3-d2fdfcfb68f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:03.481769 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93dcfa-9bea-411a-afe3-d2fdfcfb68f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:03.482039 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93dcfa-9bea-411a-afe3-d2fdfcfb68f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:03.490957 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93dcfa-9bea-411a-afe3-d2fdfcfb68f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:03.492401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93dcfa-9bea-411a-afe3-d2fdfcfb68f6 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:03.502846 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1320/2666] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:31:11.786677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=47 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:13.498687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=46 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:18.317676 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:18.319839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:18.390176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f133c5e-279a-4d52-bf29-ac59aeab9fb6 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:18.395133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:18.395735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:18.396005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:18.396273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:18.397950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:18.409670 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:18.411307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:18.421883 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1347/2667] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:31:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:31:19.100349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93dcfa-9bea-411a-afe3-d2fdfcfb68f6 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:19.168755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93dcfa-9bea-411a-afe3-d2fdfcfb68f6 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:19.173906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:19.174518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:19.174895 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:19.175211 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] PATH_INFO: `/v3/users/ad7686dcd7bb4b10ab6ff2f415613eb0` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:19.176312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7686dcd7bb4b10ab6ff2f415613eb0)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:19.178243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:19.319503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7686dcd7bb4b10ab6ff2f415613eb0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.331443 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7686dcd7bb4b10ab6ff2f415613eb0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.338448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7686dcd7bb4b10ab6ff2f415613eb0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.339178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7686dcd7bb4b10ab6ff2f415613eb0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.339782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7686dcd7bb4b10ab6ff2f415613eb0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.409968 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1321/2668] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:31:19 2022] DELETE /identity/v3/users/ad7686dcd7bb4b10ab6ff2f415613eb0 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:19.620043 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:19.631796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2778e39-492a-457a-a65c-f20481db916b service placement] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:19.691520 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:19.692146 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:19.692425 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:19.692700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] PATH_INFO: `/v3/projects/5b41a4f688ee478fb269f79ac83266d2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:19.693905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b41a4f688ee478fb269f79ac83266d2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:19.695884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:19.795262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b41a4f688ee478fb269f79ac83266d2', 'request_id': 'req-daa7a5c6-f07d-4a33-b371-f45a61dbb854'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.846731 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b41a4f688ee478fb269f79ac83266d2', 'request_id': 'req-daa7a5c6-f07d-4a33-b371-f45a61dbb854'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.852360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b41a4f688ee478fb269f79ac83266d2', 'request_id': 'req-daa7a5c6-f07d-4a33-b371-f45a61dbb854'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.863800 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] The token cache is being invalidate because project 5b41a4f688ee478fb269f79ac83266d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:31:19.864029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.864657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:19.866187 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1348/2669] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:31:19 2022] DELETE /identity/v3/projects/5b41a4f688ee478fb269f79ac83266d2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:19.870257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:19.871693 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:19.884959 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8097d-0318-4914-b91a-b1cef8ec168b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:19.941128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:19.941744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:19.942019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:19.942290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] PATH_INFO: `/v3/users/065a58a3cfbc4898a1896dc93530cf1d` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:19.943477 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=065a58a3cfbc4898a1896dc93530cf1d)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:19.945375 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:19.965936 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:20.125155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '065a58a3cfbc4898a1896dc93530cf1d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.137594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '065a58a3cfbc4898a1896dc93530cf1d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.145353 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '065a58a3cfbc4898a1896dc93530cf1d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.146144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '065a58a3cfbc4898a1896dc93530cf1d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.146758 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0ae99c7d-687b-47fa-9829-01a0f3412e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '065a58a3cfbc4898a1896dc93530cf1d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.240926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1322/2670] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:31:19 2022] DELETE /identity/v3/users/065a58a3cfbc4898a1896dc93530cf1d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:20.511492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:20.512941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:20.526306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7a5c6-f07d-4a33-b371-f45a61dbb854 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:20.544196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:20.583764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:20.584389 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:20.584671 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:20.584943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] PATH_INFO: `/v3/projects/4bcd36e2524c4adaaa3b20ba79ba5e61` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:20.586069 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bcd36e2524c4adaaa3b20ba79ba5e61)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:20.587996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:20.719160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bcd36e2524c4adaaa3b20ba79ba5e61', 'request_id': 'req-68ae4c64-3846-4df0-89fd-37f40d46d85d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.789734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bcd36e2524c4adaaa3b20ba79ba5e61', 'request_id': 'req-68ae4c64-3846-4df0-89fd-37f40d46d85d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.796295 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bcd36e2524c4adaaa3b20ba79ba5e61', 'request_id': 'req-68ae4c64-3846-4df0-89fd-37f40d46d85d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.807537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] The token cache is being invalidate because project 4bcd36e2524c4adaaa3b20ba79ba5e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:31:20.807829 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.808462 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-68ae4c64-3846-4df0-89fd-37f40d46d85d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:20.809943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1349/2671] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:31:20 2022] DELETE /identity/v3/projects/4bcd36e2524c4adaaa3b20ba79ba5e61 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:20.815027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139105-79b1-4502-b289-c933e85bdd65 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:20.815334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139105-79b1-4502-b289-c933e85bdd65 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:20.815611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139105-79b1-4502-b289-c933e85bdd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:20.835078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-7a139105-79b1-4502-b289-c933e85bdd65 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:20.836289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7a139105-79b1-4502-b289-c933e85bdd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:20.895637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1323/2672] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:20.903358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660cde-9bf0-4ae6-adc0-07de5202a279 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:20.903646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660cde-9bf0-4ae6-adc0-07de5202a279 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:20.903922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660cde-9bf0-4ae6-adc0-07de5202a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:20.924016 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e7660cde-9bf0-4ae6-adc0-07de5202a279 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:20.925259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e7660cde-9bf0-4ae6-adc0-07de5202a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:20.932941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1350/2673] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:20.937029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a139105-79b1-4502-b289-c933e85bdd65 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:20.946485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a139105-79b1-4502-b289-c933e85bdd65 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:20.950913 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:20.952025 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:20.952400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:20.952671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:20.953304 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:31:20.953614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:20.955203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:20.966247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1324/2674] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:31:20.970910 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7660cde-9bf0-4ae6-adc0-07de5202a279 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:20.979898 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7660cde-9bf0-4ae6-adc0-07de5202a279 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:20.984638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8fc98-38e6-4f2f-8f4d-0348cbf675cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:20.985224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8fc98-38e6-4f2f-8f4d-0348cbf675cd admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:20.985501 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8fc98-38e6-4f2f-8f4d-0348cbf675cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:20.985770 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8fc98-38e6-4f2f-8f4d-0348cbf675cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:20.986381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8fc98-38e6-4f2f-8f4d-0348cbf675cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:20.988271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8fc98-38e6-4f2f-8f4d-0348cbf675cd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:21.111570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1351/2675] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:31:20 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:31:21.115720 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:21.126301 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd621f-f830-4c87-aeb7-f5f127188e8e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:21.182854 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f3cf7-5ca6-4776-87ac-9de30ca2a2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:21.183694 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f3cf7-5ca6-4776-87ac-9de30ca2a2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:21.183960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f3cf7-5ca6-4776-87ac-9de30ca2a2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:21.184227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f3cf7-5ca6-4776-87ac-9de30ca2a2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:21.184873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f3cf7-5ca6-4776-87ac-9de30ca2a2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:21.186995 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f3cf7-5ca6-4776-87ac-9de30ca2a2f6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:21.261009 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1325/2676] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:31:21 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:31:21.266029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8fc98-38e6-4f2f-8f4d-0348cbf675cd admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:21.277317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8fc98-38e6-4f2f-8f4d-0348cbf675cd admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:21.282224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea967a4b-e80b-4649-81d8-33a7f6de7882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:21.286981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967a4b-e80b-4649-81d8-33a7f6de7882 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:21.286981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967a4b-e80b-4649-81d8-33a7f6de7882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:21.286981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967a4b-e80b-4649-81d8-33a7f6de7882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:21.286981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea967a4b-e80b-4649-81d8-33a7f6de7882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:21.286981 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea967a4b-e80b-4649-81d8-33a7f6de7882 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:21.296259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1352/2677] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:31:21.300086 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f3cf7-5ca6-4776-87ac-9de30ca2a2f6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:21.310224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f3cf7-5ca6-4776-87ac-9de30ca2a2f6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:21.315829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963dcb1-3980-4f21-b5a1-7063067ff0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:21.316466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963dcb1-3980-4f21-b5a1-7063067ff0ae admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:21.316749 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963dcb1-3980-4f21-b5a1-7063067ff0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:21.317023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963dcb1-3980-4f21-b5a1-7063067ff0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:21.317664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963dcb1-3980-4f21-b5a1-7063067ff0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:21.319296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963dcb1-3980-4f21-b5a1-7063067ff0ae admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:21.330709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1326/2678] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:31:21.334725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea967a4b-e80b-4649-81d8-33a7f6de7882 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:21.344771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea967a4b-e80b-4649-81d8-33a7f6de7882 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:21.349350 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:21.349986 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:21.350262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:21.350535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] PATH_INFO: `/v3/projects/ef644f1f4ce74cbc8b1bb1113c96f7a7/users/254ad748b2dd45e8bc98e615620d64c2/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:21.365143 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef644f1f4ce74cbc8b1bb1113c96f7a7, user_id=254ad748b2dd45e8bc98e615620d64c2, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:21.367732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:21.465962 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1353/2679] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:31:21 2022] PUT /identity/v3/projects/ef644f1f4ce74cbc8b1bb1113c96f7a7/users/254ad748b2dd45e8bc98e615620d64c2/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:21.477831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a9e1-c662-45f4-817b-b11cfe3711f3 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:21.478120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a9e1-c662-45f4-817b-b11cfe3711f3 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:21.478394 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a9e1-c662-45f4-817b-b11cfe3711f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:21.486481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:21.506530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3a99a9e1-c662-45f4-817b-b11cfe3711f3 None None] MFA Rules not processed for user `254ad748b2dd45e8bc98e615620d64c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:21.507773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3a99a9e1-c662-45f4-817b-b11cfe3711f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:21.572660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1327/2680] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:21.584434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:21.586141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:21.638452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:21.643911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe93f2-e506-44ad-9392-29faa0985878 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:21.647524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5cbf8-2d24-4b92-ba64-6723e067c07d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:21.648118 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5cbf8-2d24-4b92-ba64-6723e067c07d service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:21.651546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5cbf8-2d24-4b92-ba64-6723e067c07d service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:21.651546 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5cbf8-2d24-4b92-ba64-6723e067c07d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:21.657065 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5cbf8-2d24-4b92-ba64-6723e067c07d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:21.658484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5cbf8-2d24-4b92-ba64-6723e067c07d service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:21.667124 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1354/2681] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:31:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:31:31.612619 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=30 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:31.757362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:45.558302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99a9e1-c662-45f4-817b-b11cfe3711f3 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:45.619044 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99a9e1-c662-45f4-817b-b11cfe3711f3 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:45.623523 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:45.624145 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:45.624420 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:45.625212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] PATH_INFO: `/v3/users/254ad748b2dd45e8bc98e615620d64c2` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:45.626377 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=254ad748b2dd45e8bc98e615620d64c2)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:45.628310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:45.802210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254ad748b2dd45e8bc98e615620d64c2'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:45.814565 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254ad748b2dd45e8bc98e615620d64c2'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:45.821315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254ad748b2dd45e8bc98e615620d64c2'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:45.824808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254ad748b2dd45e8bc98e615620d64c2'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:45.824808 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254ad748b2dd45e8bc98e615620d64c2'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:45.905418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1328/2682] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:31:45 2022] DELETE /identity/v3/users/254ad748b2dd45e8bc98e615620d64c2 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:45.910027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570b3e4-0314-4dbd-9756-ef5dfdc3b670 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:45.916388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570b3e4-0314-4dbd-9756-ef5dfdc3b670 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:45.916388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570b3e4-0314-4dbd-9756-ef5dfdc3b670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:45.929213 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c570b3e4-0314-4dbd-9756-ef5dfdc3b670 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:45.930419 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c570b3e4-0314-4dbd-9756-ef5dfdc3b670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:45.988613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1355/2683] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:45.998135 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:45.999597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:46.055570 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289228d-ab86-4db8-90fa-cb82c639d121 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:46.059024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5b672-99ce-4c0f-b88f-e8141936b464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:46.059618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b672-99ce-4c0f-b88f-e8141936b464 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:46.059894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b672-99ce-4c0f-b88f-e8141936b464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:46.060161 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b672-99ce-4c0f-b88f-e8141936b464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:46.069527 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5b672-99ce-4c0f-b88f-e8141936b464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:46.070944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5b672-99ce-4c0f-b88f-e8141936b464 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:46.080428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1329/2684] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:31:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:31:46.499387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570b3e4-0314-4dbd-9756-ef5dfdc3b670 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:46.501001 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c570b3e4-0314-4dbd-9756-ef5dfdc3b670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:46.513162 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570b3e4-0314-4dbd-9756-ef5dfdc3b670 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:46.518429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:46.519100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:46.519388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:46.519704 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] PATH_INFO: `/v3/projects/ef644f1f4ce74cbc8b1bb1113c96f7a7` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:46.520798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef644f1f4ce74cbc8b1bb1113c96f7a7)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:46.522699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:46.623634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef644f1f4ce74cbc8b1bb1113c96f7a7', 'request_id': 'req-0253cbf7-90f1-4041-9549-1df15724c6dd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:46.694331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef644f1f4ce74cbc8b1bb1113c96f7a7', 'request_id': 'req-0253cbf7-90f1-4041-9549-1df15724c6dd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:46.700262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef644f1f4ce74cbc8b1bb1113c96f7a7', 'request_id': 'req-0253cbf7-90f1-4041-9549-1df15724c6dd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:46.711297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] The token cache is being invalidate because project ef644f1f4ce74cbc8b1bb1113c96f7a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:31:46.711532 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:46.712163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0253cbf7-90f1-4041-9549-1df15724c6dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:31:46.713646 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1356/2685] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:31:46 2022] DELETE /identity/v3/projects/ef644f1f4ce74cbc8b1bb1113c96f7a7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:46.719378 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e42cf9-9700-4aa0-8d0c-8a462f370799 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:46.719879 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e42cf9-9700-4aa0-8d0c-8a462f370799 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:46.719989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e42cf9-9700-4aa0-8d0c-8a462f370799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:46.739772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-d8e42cf9-9700-4aa0-8d0c-8a462f370799 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:46.740960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d8e42cf9-9700-4aa0-8d0c-8a462f370799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:46.757530 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:46.799503 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1330/2686] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:46.807235 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88119905-0899-4fbb-bbe3-61a5c4c846a7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:46.807527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88119905-0899-4fbb-bbe3-61a5c4c846a7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:46.807806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88119905-0899-4fbb-bbe3-61a5c4c846a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:46.829024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-88119905-0899-4fbb-bbe3-61a5c4c846a7 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:46.830216 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-88119905-0899-4fbb-bbe3-61a5c4c846a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:46.838121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1357/2687] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:46.842344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e42cf9-9700-4aa0-8d0c-8a462f370799 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:46.852914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e42cf9-9700-4aa0-8d0c-8a462f370799 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:46.857463 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:46.858090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:46.858366 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:46.858637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:46.859335 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:31:46.859631 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:46.861208 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:46.873226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1331/2688] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:31:46.877819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88119905-0899-4fbb-bbe3-61a5c4c846a7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:46.886943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88119905-0899-4fbb-bbe3-61a5c4c846a7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:46.892430 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc29d60-d50f-4cb4-82b8-8ddd8b747b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:46.893018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc29d60-d50f-4cb4-82b8-8ddd8b747b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:46.893297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc29d60-d50f-4cb4-82b8-8ddd8b747b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:46.893569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc29d60-d50f-4cb4-82b8-8ddd8b747b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:46.894196 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc29d60-d50f-4cb4-82b8-8ddd8b747b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:46.896107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc29d60-d50f-4cb4-82b8-8ddd8b747b64 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:46.986630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1358/2689] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:31:46 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:31:46.990796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:47.000796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefacce5-19a9-4483-9d84-e9edec5c9f18 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:47.057437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff15080-2d6f-4961-b864-09d3d39065f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:47.058029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff15080-2d6f-4961-b864-09d3d39065f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:47.058307 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff15080-2d6f-4961-b864-09d3d39065f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:47.058583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff15080-2d6f-4961-b864-09d3d39065f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:47.059235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff15080-2d6f-4961-b864-09d3d39065f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:47.061132 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff15080-2d6f-4961-b864-09d3d39065f7 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:47.140165 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1332/2690] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:31:46 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:31:47.144922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc29d60-d50f-4cb4-82b8-8ddd8b747b64 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:47.155302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc29d60-d50f-4cb4-82b8-8ddd8b747b64 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:47.160181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f30d32-8f6a-4432-9178-44cdb5e0ce6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:47.160772 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30d32-8f6a-4432-9178-44cdb5e0ce6c admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:47.161079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30d32-8f6a-4432-9178-44cdb5e0ce6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:47.161357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30d32-8f6a-4432-9178-44cdb5e0ce6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:47.162009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f30d32-8f6a-4432-9178-44cdb5e0ce6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:47.163623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f30d32-8f6a-4432-9178-44cdb5e0ce6c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:47.173514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1359/2691] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:31:47.180833 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff15080-2d6f-4961-b864-09d3d39065f7 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:47.191496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff15080-2d6f-4961-b864-09d3d39065f7 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:47.196240 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4fdd7-9acb-4226-8b3c-de7320b771d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:47.196961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fdd7-9acb-4226-8b3c-de7320b771d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:47.197252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fdd7-9acb-4226-8b3c-de7320b771d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:47.197546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fdd7-9acb-4226-8b3c-de7320b771d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:47.198191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4fdd7-9acb-4226-8b3c-de7320b771d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:47.199820 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4fdd7-9acb-4226-8b3c-de7320b771d0 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:47.211014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1333/2692] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:31:47.214960 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f30d32-8f6a-4432-9178-44cdb5e0ce6c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:47.224884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f30d32-8f6a-4432-9178-44cdb5e0ce6c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:47.229379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:47.230044 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:47.230326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:47.230599 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] PATH_INFO: `/v3/projects/0e2aa26b8ec54387b7c19fc0a2b9438f/users/6c52cd52d25a42e29c8f049c94c903aa/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:47.244872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2aa26b8ec54387b7c19fc0a2b9438f, user_id=6c52cd52d25a42e29c8f049c94c903aa, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:47.247511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:47.350768 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1360/2693] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:31:47 2022] PUT /identity/v3/projects/0e2aa26b8ec54387b7c19fc0a2b9438f/users/6c52cd52d25a42e29c8f049c94c903aa/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:31:47.355752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179bb31-5d60-45c7-916c-a79f76a86265 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:47.356050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179bb31-5d60-45c7-916c-a79f76a86265 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:47.356329 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179bb31-5d60-45c7-916c-a79f76a86265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:47.375395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c179bb31-5d60-45c7-916c-a79f76a86265 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:47.376598 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c179bb31-5d60-45c7-916c-a79f76a86265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:47.434024 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1334/2694] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:47.443975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:47.445693 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:47.459838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:47.503374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976a42-9f6a-4a9f-a465-b3361b6789de admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:47.507079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba681bb-5cb3-46ea-a7d1-f74f6d719bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:47.507681 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba681bb-5cb3-46ea-a7d1-f74f6d719bce service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:47.507963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba681bb-5cb3-46ea-a7d1-f74f6d719bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:47.508237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba681bb-5cb3-46ea-a7d1-f74f6d719bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:47.517978 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba681bb-5cb3-46ea-a7d1-f74f6d719bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:47.519438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba681bb-5cb3-46ea-a7d1-f74f6d719bce service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:47.528643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1361/2695] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:31:52.660379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4b83f-5278-4033-b752-c1fe232c8d26 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:52.660937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4b83f-5278-4033-b752-c1fe232c8d26 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:52.660937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4b83f-5278-4033-b752-c1fe232c8d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:52.690759 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-4ad4b83f-5278-4033-b752-c1fe232c8d26 None None] MFA Rules not processed for user `6c52cd52d25a42e29c8f049c94c903aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:52.692187 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4ad4b83f-5278-4033-b752-c1fe232c8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:52.710485 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:52.760294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1335/2696] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:52.765477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba681bb-5cb3-46ea-a7d1-f74f6d719bce service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:52.776243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba681bb-5cb3-46ea-a7d1-f74f6d719bce service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:52.781605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44034c5a-2e78-4e4a-8ec5-9398f9046169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:52.782154 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44034c5a-2e78-4e4a-8ec5-9398f9046169 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:52.782435 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44034c5a-2e78-4e4a-8ec5-9398f9046169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:52.782710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44034c5a-2e78-4e4a-8ec5-9398f9046169 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:52.783388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44034c5a-2e78-4e4a-8ec5-9398f9046169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:52.785301 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44034c5a-2e78-4e4a-8ec5-9398f9046169 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:52.884650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1362/2697] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:31:52 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:31:52.888682 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4b83f-5278-4033-b752-c1fe232c8d26 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:52.900678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4b83f-5278-4033-b752-c1fe232c8d26 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:52.958841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025efd6-589e-48bb-8659-abd8592cb132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:52.959493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025efd6-589e-48bb-8659-abd8592cb132 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:52.959772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025efd6-589e-48bb-8659-abd8592cb132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:52.960097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025efd6-589e-48bb-8659-abd8592cb132 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:52.960724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025efd6-589e-48bb-8659-abd8592cb132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:52.962636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025efd6-589e-48bb-8659-abd8592cb132 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:53.020974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1336/2698] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:31:52 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:31:53.025240 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44034c5a-2e78-4e4a-8ec5-9398f9046169 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:53.037027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44034c5a-2e78-4e4a-8ec5-9398f9046169 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:53.042104 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9bbcf-d23f-4633-8f4b-b6ede8b69264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:53.042817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9bbcf-d23f-4633-8f4b-b6ede8b69264 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:53.043121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9bbcf-d23f-4633-8f4b-b6ede8b69264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:53.043407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9bbcf-d23f-4633-8f4b-b6ede8b69264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:53.044064 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9bbcf-d23f-4633-8f4b-b6ede8b69264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:53.045722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9bbcf-d23f-4633-8f4b-b6ede8b69264 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:53.060398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1363/2699] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:31:53.064974 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025efd6-589e-48bb-8659-abd8592cb132 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:53.075732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025efd6-589e-48bb-8659-abd8592cb132 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:53.081316 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb7be5-c047-4427-b597-84d999faf5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:53.081966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb7be5-c047-4427-b597-84d999faf5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:53.082248 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb7be5-c047-4427-b597-84d999faf5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:53.082524 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb7be5-c047-4427-b597-84d999faf5bc admin admin] PATH_INFO: `/v3/projects/bcd2cae596b94d399c13b72db54e1406/users/7ce4d8dd8b4143ca90a9ecddaa8603b5/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:53.098153 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb7be5-c047-4427-b597-84d999faf5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd2cae596b94d399c13b72db54e1406, user_id=7ce4d8dd8b4143ca90a9ecddaa8603b5, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:53.100865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb7be5-c047-4427-b597-84d999faf5bc admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:53.184072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1337/2700] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:31:53 2022] PUT /identity/v3/projects/bcd2cae596b94d399c13b72db54e1406/users/7ce4d8dd8b4143ca90a9ecddaa8603b5/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:53.188710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9bbcf-d23f-4633-8f4b-b6ede8b69264 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:53.198458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9bbcf-d23f-4633-8f4b-b6ede8b69264 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:53.256393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3bf25-d452-48ac-b20b-22e63da8e12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:53.256996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3bf25-d452-48ac-b20b-22e63da8e12f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:53.257275 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3bf25-d452-48ac-b20b-22e63da8e12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:53.257551 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3bf25-d452-48ac-b20b-22e63da8e12f admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:53.258195 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3bf25-d452-48ac-b20b-22e63da8e12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:53.259804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3bf25-d452-48ac-b20b-22e63da8e12f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:53.271297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1364/2701] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:31:53.275442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb7be5-c047-4427-b597-84d999faf5bc admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:53.285538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb7be5-c047-4427-b597-84d999faf5bc admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:53.290110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:53.290689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:53.290971 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:53.291272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] PATH_INFO: `/v3/projects/bcd2cae596b94d399c13b72db54e1406/users/7ce4d8dd8b4143ca90a9ecddaa8603b5/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:53.293129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd2cae596b94d399c13b72db54e1406, user_id=7ce4d8dd8b4143ca90a9ecddaa8603b5, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:53.295873 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:53.305840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1338/2702] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:31:53 2022] PUT /identity/v3/projects/bcd2cae596b94d399c13b72db54e1406/users/7ce4d8dd8b4143ca90a9ecddaa8603b5/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:31:58.569720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495fe19-635e-41e1-acf3-8acb56613020 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:58.570626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495fe19-635e-41e1-acf3-8acb56613020 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:58.570907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495fe19-635e-41e1-acf3-8acb56613020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:58.601562 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2495fe19-635e-41e1-acf3-8acb56613020 None None] MFA Rules not processed for user `7ce4d8dd8b4143ca90a9ecddaa8603b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:31:58.602802 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2495fe19-635e-41e1-acf3-8acb56613020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:58.681257 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1365/2703] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:31:58.700452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:31:58.712627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:31:58.801285 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa5040-ea49-47da-8474-84cd1f7d85a0 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:31:58.806923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d128f7-a8d4-42b4-8cc9-4e52bc633474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:31:58.809946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d128f7-a8d4-42b4-8cc9-4e52bc633474 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:31:58.809946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d128f7-a8d4-42b4-8cc9-4e52bc633474 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:31:58.809946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d128f7-a8d4-42b4-8cc9-4e52bc633474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:31:58.814205 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:31:58.870920 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d128f7-a8d4-42b4-8cc9-4e52bc633474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:31:58.872626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d128f7-a8d4-42b4-8cc9-4e52bc633474 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:31:58.882702 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1339/2704] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:31:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:32:08.677697 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:32:08.897397 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=24 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:32:16.715565 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495fe19-635e-41e1-acf3-8acb56613020 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:32:16.716977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2495fe19-635e-41e1-acf3-8acb56613020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:16.774539 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495fe19-635e-41e1-acf3-8acb56613020 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:32:16.778116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:32:16.778716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:32:16.778994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:32:16.779293 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:32:16.780966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:16.795056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:32:16.796472 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:32:16.807356 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1366/2705] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:32:18.405747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547fb58-5f50-473d-be1c-5b6cee10ec4c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:32:18.406300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547fb58-5f50-473d-be1c-5b6cee10ec4c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:32:18.406300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547fb58-5f50-473d-be1c-5b6cee10ec4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:32:18.425388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-0547fb58-5f50-473d-be1c-5b6cee10ec4c None None] MFA Rules not processed for user `6c52cd52d25a42e29c8f049c94c903aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:32:18.426585 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0547fb58-5f50-473d-be1c-5b6cee10ec4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:18.433352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1340/2706] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:32:18.472158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:32:18.473677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:18.484415 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946d2ff-6be6-4a41-b5c0-7d70ed8dc598 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:32:18.493712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:32:18.493712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:32:18.493712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:32:18.493712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:32:18.494113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:18.505385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:32:18.506791 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:32:18.517900 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1367/2707] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:32:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:32:19.640313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547fb58-5f50-473d-be1c-5b6cee10ec4c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:32:19.641841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0547fb58-5f50-473d-be1c-5b6cee10ec4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:19.718022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547fb58-5f50-473d-be1c-5b6cee10ec4c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:32:19.723581 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:32:19.724180 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:32:19.724456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:32:19.724724 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:32:19.726425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:19.737723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:32:19.739315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:32:19.748841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1341/2708] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:32:22.993135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:32:22.993135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:23.082174 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212f038-f41a-4f1b-b090-1592ca83fb6b service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:32:23.087337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:32:23.088056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:32:23.088923 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:32:23.089635 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:32:23.091913 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:23.106083 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:32:23.108107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:32:23.117045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1368/2709] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:32:24.090945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:32:24.092552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:24.112966 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69681f-09d0-40f7-9788-6a46768fafd7 service placement] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:32:24.119595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:32:24.121762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:32:24.122325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:32:24.122800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:32:24.125692 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:24.141939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:32:24.143714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:32:24.154771 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1342/2710] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:32:33.113067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=58 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:32:34.150594 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=41 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:32:36.322723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:32:36.331184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03188c2d-4152-4e76-9998-6ad6f60deb96 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:32:36.334852 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45b39a-30df-44b4-a0a9-efde43949421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:32:36.335469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b39a-30df-44b4-a0a9-efde43949421 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:32:36.335747 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b39a-30df-44b4-a0a9-efde43949421 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:32:36.336017 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b39a-30df-44b4-a0a9-efde43949421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:32:36.344752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45b39a-30df-44b4-a0a9-efde43949421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:32:36.346156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45b39a-30df-44b4-a0a9-efde43949421 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:32:36.355582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1369/2711] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:32:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:32:38.285345 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:32:38.296237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e30fc-39fd-445e-b3eb-78dcc1c9eca8 service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:32:38.302056 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:32:38.302645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:32:38.302926 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:32:38.303226 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:32:38.304924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:32:38.317934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:32:38.319540 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:32:38.330166 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1343/2712] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:32:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:32:46.351454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:32:48.325735 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:33:15.237604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6703300-4b8c-43ce-a134-89f8a6994c73 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:33:15.238132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6703300-4b8c-43ce-a134-89f8a6994c73 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:33:15.238132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6703300-4b8c-43ce-a134-89f8a6994c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:33:15.257712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a6703300-4b8c-43ce-a134-89f8a6994c73 None None] MFA Rules not processed for user `20f7cd1ed1214372a5c6f441fb98f799`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:33:15.258928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6703300-4b8c-43ce-a134-89f8a6994c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:33:15.313642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1370/2713] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:33:15.610660 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:33:15.619848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9ca77-a696-4ea6-94fb-ac6df03d2f5e service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:33:15.623363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10954440-0095-4c44-93b6-c9e79067bff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:33:15.623947 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954440-0095-4c44-93b6-c9e79067bff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:33:15.624224 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954440-0095-4c44-93b6-c9e79067bff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:33:15.624499 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954440-0095-4c44-93b6-c9e79067bff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:33:15.626177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-10954440-0095-4c44-93b6-c9e79067bff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:33:15.635370 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10954440-0095-4c44-93b6-c9e79067bff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:33:15.636807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10954440-0095-4c44-93b6-c9e79067bff9 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:33:15.644725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1344/2714] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:33:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:33:23.948060 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6703300-4b8c-43ce-a134-89f8a6994c73 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:33:23.961778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6703300-4b8c-43ce-a134-89f8a6994c73 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:33:23.967324 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:33:23.967922 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:33:23.968204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:33:23.968476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:33:23.970407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:33:23.983951 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:33:23.985531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:33:23.997220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1371/2715] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:33:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:33:25.641284 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:33:33.992594 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=31 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:34:41.495867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcff3e-766e-45f3-84e1-b014c23c1367 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:41.496373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcff3e-766e-45f3-84e1-b014c23c1367 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:41.496476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcff3e-766e-45f3-84e1-b014c23c1367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:41.515905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-19bcff3e-766e-45f3-84e1-b014c23c1367 None None] MFA Rules not processed for user `6c52cd52d25a42e29c8f049c94c903aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:34:41.517097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-19bcff3e-766e-45f3-84e1-b014c23c1367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:34:41.525347 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1345/2716] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:34:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:34:41.841459 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:41.851136 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07643aa4-dff3-4a8c-aa5d-7dc1253dad7b service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:41.854910 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:41.855524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:41.855801 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:41.856077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:41.857774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:34:41.867442 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:41.869762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:41.881421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1372/2717] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:34:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:34:46.725950 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcff3e-766e-45f3-84e1-b014c23c1367 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:46.728242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-19bcff3e-766e-45f3-84e1-b014c23c1367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:34:46.801556 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcff3e-766e-45f3-84e1-b014c23c1367 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:46.806264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:46.806911 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:46.807330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:46.807645 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] PATH_INFO: `/v3/users/20f7cd1ed1214372a5c6f441fb98f799` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:46.808798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f7cd1ed1214372a5c6f441fb98f799)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:46.810842 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:46.995289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f7cd1ed1214372a5c6f441fb98f799'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.008104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f7cd1ed1214372a5c6f441fb98f799'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.014897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f7cd1ed1214372a5c6f441fb98f799'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.015899 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f7cd1ed1214372a5c6f441fb98f799'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.016506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-35b94f23-ee94-4e16-9fe2-f2e726e3ad44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f7cd1ed1214372a5c6f441fb98f799'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.089784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1346/2718] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:34:46 2022] DELETE /identity/v3/users/20f7cd1ed1214372a5c6f441fb98f799 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:34:47.372716 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:47.388191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72d94d-1d9a-4292-91d7-99010f205c5a service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:47.459199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:47.459860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:47.460139 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:47.460408 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] PATH_INFO: `/v3/projects/a311cd62403a4fa285c48a145dd9ccfb` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:47.461540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a311cd62403a4fa285c48a145dd9ccfb)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:47.463634 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:47.566545 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a311cd62403a4fa285c48a145dd9ccfb', 'request_id': 'req-84be3949-eb64-4f10-b2a3-0e5538cb20cc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.627476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a311cd62403a4fa285c48a145dd9ccfb', 'request_id': 'req-84be3949-eb64-4f10-b2a3-0e5538cb20cc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.633813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a311cd62403a4fa285c48a145dd9ccfb', 'request_id': 'req-84be3949-eb64-4f10-b2a3-0e5538cb20cc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.645535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] The token cache is being invalidate because project a311cd62403a4fa285c48a145dd9ccfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:34:47.645817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.646538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84be3949-eb64-4f10-b2a3-0e5538cb20cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:34:47.648041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1373/2719] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:34:47 2022] DELETE /identity/v3/projects/a311cd62403a4fa285c48a145dd9ccfb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:34:47.653057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf74a5-49d6-49a0-bc42-2e36ddc8f9a6 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:47.653346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf74a5-49d6-49a0-bc42-2e36ddc8f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:47.653623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf74a5-49d6-49a0-bc42-2e36ddc8f9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:47.673254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-cddf74a5-49d6-49a0-bc42-2e36ddc8f9a6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:34:47.674457 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cddf74a5-49d6-49a0-bc42-2e36ddc8f9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:34:47.735066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1347/2720] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:34:47.744859 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1193d-9c33-49e7-be93-4f0f7b606b86 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:47.745148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1193d-9c33-49e7-be93-4f0f7b606b86 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:47.745422 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1193d-9c33-49e7-be93-4f0f7b606b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:47.765846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-98b1193d-9c33-49e7-be93-4f0f7b606b86 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:34:47.767102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-98b1193d-9c33-49e7-be93-4f0f7b606b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:34:47.775337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1374/2721] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:34:47.779497 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf74a5-49d6-49a0-bc42-2e36ddc8f9a6 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:47.789537 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf74a5-49d6-49a0-bc42-2e36ddc8f9a6 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:47.794058 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:47.794646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:47.794929 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:47.795234 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:47.799751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:34:47.799751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:47.799751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:47.807189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:34:47.810330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1348/2722] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:34:47.814699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1193d-9c33-49e7-be93-4f0f7b606b86 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:47.824763 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1193d-9c33-49e7-be93-4f0f7b606b86 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:47.829753 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c5cee-5c06-4d90-ae13-a95f31cf1b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:47.830359 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5cee-5c06-4d90-ae13-a95f31cf1b56 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:47.830642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5cee-5c06-4d90-ae13-a95f31cf1b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:47.830943 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5cee-5c06-4d90-ae13-a95f31cf1b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:47.831607 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c5cee-5c06-4d90-ae13-a95f31cf1b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:47.833714 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c5cee-5c06-4d90-ae13-a95f31cf1b56 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:47.916862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1375/2723] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:34:47 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:34:47.920890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:47.931691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82469efd-3c68-424f-84a6-3fa8b1f485d0 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:47.986673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17646129-1368-4d7a-b2cc-dcf80ced54aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:47.987236 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646129-1368-4d7a-b2cc-dcf80ced54aa admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:47.987522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646129-1368-4d7a-b2cc-dcf80ced54aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:47.987797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646129-1368-4d7a-b2cc-dcf80ced54aa admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:47.988425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17646129-1368-4d7a-b2cc-dcf80ced54aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:47.990509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17646129-1368-4d7a-b2cc-dcf80ced54aa admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.061592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1349/2724] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:34:47 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:34:48.065642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c5cee-5c06-4d90-ae13-a95f31cf1b56 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.076000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c5cee-5c06-4d90-ae13-a95f31cf1b56 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.080578 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fda06-357a-4c19-9e69-d37b0c764167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.081173 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fda06-357a-4c19-9e69-d37b0c764167 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.081456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fda06-357a-4c19-9e69-d37b0c764167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.081736 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fda06-357a-4c19-9e69-d37b0c764167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.082405 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fda06-357a-4c19-9e69-d37b0c764167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.084926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fda06-357a-4c19-9e69-d37b0c764167 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.095306 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1376/2725] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:34:48.099940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17646129-1368-4d7a-b2cc-dcf80ced54aa admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.110163 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17646129-1368-4d7a-b2cc-dcf80ced54aa admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.114799 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbac1fe-871d-41fb-9719-7ae59ef0cea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.115435 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbac1fe-871d-41fb-9719-7ae59ef0cea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.115716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbac1fe-871d-41fb-9719-7ae59ef0cea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.115993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbac1fe-871d-41fb-9719-7ae59ef0cea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.116636 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbac1fe-871d-41fb-9719-7ae59ef0cea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.118235 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbac1fe-871d-41fb-9719-7ae59ef0cea8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.133369 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1350/2726] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:34:48.138032 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fda06-357a-4c19-9e69-d37b0c764167 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.148754 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fda06-357a-4c19-9e69-d37b0c764167 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.153379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815ac4f-5677-4141-9326-8c0dcfae4b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.154029 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815ac4f-5677-4141-9326-8c0dcfae4b88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.154309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815ac4f-5677-4141-9326-8c0dcfae4b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.154586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815ac4f-5677-4141-9326-8c0dcfae4b88 admin admin] PATH_INFO: `/v3/projects/33d779180933436ca676e19c0ec8f35c/users/eaa89695c37f40adb935978faceb662d/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.169881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815ac4f-5677-4141-9326-8c0dcfae4b88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d779180933436ca676e19c0ec8f35c, user_id=eaa89695c37f40adb935978faceb662d, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.172610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815ac4f-5677-4141-9326-8c0dcfae4b88 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.272994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1377/2727] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:34:48 2022] PUT /identity/v3/projects/33d779180933436ca676e19c0ec8f35c/users/eaa89695c37f40adb935978faceb662d/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:34:48.283012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f837e2-24ce-45c0-a38c-7bcf567ece07 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.283311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f837e2-24ce-45c0-a38c-7bcf567ece07 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.283588 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f837e2-24ce-45c0-a38c-7bcf567ece07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.310797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-38f837e2-24ce-45c0-a38c-7bcf567ece07 None None] MFA Rules not processed for user `eaa89695c37f40adb935978faceb662d`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:34:48.312057 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-38f837e2-24ce-45c0-a38c-7bcf567ece07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:34:48.373432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1351/2728] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:34:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:34:48.377837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815ac4f-5677-4141-9326-8c0dcfae4b88 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.392048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:34:48.440163 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815ac4f-5677-4141-9326-8c0dcfae4b88 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.445397 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e7cf9-8103-4136-89f0-a9a34410173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.445987 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e7cf9-8103-4136-89f0-a9a34410173f admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.446265 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e7cf9-8103-4136-89f0-a9a34410173f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.446535 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e7cf9-8103-4136-89f0-a9a34410173f admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.447282 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e7cf9-8103-4136-89f0-a9a34410173f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.449185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e7cf9-8103-4136-89f0-a9a34410173f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.566484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1378/2729] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:34:48 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:34:48.571101 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f837e2-24ce-45c0-a38c-7bcf567ece07 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.581796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f837e2-24ce-45c0-a38c-7bcf567ece07 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.639548 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824a508-a831-41f4-a915-deaaba810194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.640149 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824a508-a831-41f4-a915-deaaba810194 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.640689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824a508-a831-41f4-a915-deaaba810194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.640958 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824a508-a831-41f4-a915-deaaba810194 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.641591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824a508-a831-41f4-a915-deaaba810194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.643493 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824a508-a831-41f4-a915-deaaba810194 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.690461 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:34:48.697150 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1352/2730] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:34:48 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:34:48.702317 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e7cf9-8103-4136-89f0-a9a34410173f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.711757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e7cf9-8103-4136-89f0-a9a34410173f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.716469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992bac3-2d1d-421f-a317-3ed91c757434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.717053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992bac3-2d1d-421f-a317-3ed91c757434 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.717384 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992bac3-2d1d-421f-a317-3ed91c757434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.717653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992bac3-2d1d-421f-a317-3ed91c757434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.718318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992bac3-2d1d-421f-a317-3ed91c757434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.719928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992bac3-2d1d-421f-a317-3ed91c757434 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.730638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1379/2731] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:34:48.734716 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824a508-a831-41f4-a915-deaaba810194 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.745642 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824a508-a831-41f4-a915-deaaba810194 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.750428 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7ce49-1f59-4dda-be05-435459c2ba2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.751090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7ce49-1f59-4dda-be05-435459c2ba2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.751368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7ce49-1f59-4dda-be05-435459c2ba2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.751638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7ce49-1f59-4dda-be05-435459c2ba2f admin admin] PATH_INFO: `/v3/projects/e9f9f426cbeb4b5aa2d58eee35f7e54e/users/1ffa30cd80ad45918edbff5d466d7c3b/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.767120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7ce49-1f59-4dda-be05-435459c2ba2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f9f426cbeb4b5aa2d58eee35f7e54e, user_id=1ffa30cd80ad45918edbff5d466d7c3b, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.769826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7ce49-1f59-4dda-be05-435459c2ba2f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.878621 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1353/2732] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:34:48 2022] PUT /identity/v3/projects/e9f9f426cbeb4b5aa2d58eee35f7e54e/users/1ffa30cd80ad45918edbff5d466d7c3b/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:34:48.883429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992bac3-2d1d-421f-a317-3ed91c757434 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.893816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992bac3-2d1d-421f-a317-3ed91c757434 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.949665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711d8de-2461-44a6-bd4c-b64bb0154fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.950264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d8de-2461-44a6-bd4c-b64bb0154fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.950547 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d8de-2461-44a6-bd4c-b64bb0154fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.950824 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d8de-2461-44a6-bd4c-b64bb0154fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.951498 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711d8de-2461-44a6-bd4c-b64bb0154fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.953082 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711d8de-2461-44a6-bd4c-b64bb0154fb2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.962746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1380/2733] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:34:48.966831 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7ce49-1f59-4dda-be05-435459c2ba2f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:48.977214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7ce49-1f59-4dda-be05-435459c2ba2f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:48.981664 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:48.982294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:48.982569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:48.982837 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] PATH_INFO: `/v3/projects/e9f9f426cbeb4b5aa2d58eee35f7e54e/users/1ffa30cd80ad45918edbff5d466d7c3b/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:48.984741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f9f426cbeb4b5aa2d58eee35f7e54e, user_id=1ffa30cd80ad45918edbff5d466d7c3b, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:48.987372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:48.997738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1354/2734] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:34:48 2022] PUT /identity/v3/projects/e9f9f426cbeb4b5aa2d58eee35f7e54e/users/1ffa30cd80ad45918edbff5d466d7c3b/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:34:49.009798 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309dbe8-3ff1-400e-bbe2-4602a5d4b6b1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:49.010087 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309dbe8-3ff1-400e-bbe2-4602a5d4b6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:49.010368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309dbe8-3ff1-400e-bbe2-4602a5d4b6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:49.037507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a309dbe8-3ff1-400e-bbe2-4602a5d4b6b1 None None] MFA Rules not processed for user `1ffa30cd80ad45918edbff5d466d7c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:34:49.038722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a309dbe8-3ff1-400e-bbe2-4602a5d4b6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:34:49.106080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1381/2735] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:34:49.121305 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:49.122979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:34:49.180232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a78ebc-c170-44fa-8652-2910511a0eaa admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:49.183760 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e779878-a547-4c02-8b0b-3d1facce350f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:49.184363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e779878-a547-4c02-8b0b-3d1facce350f service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:49.184641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e779878-a547-4c02-8b0b-3d1facce350f service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:49.184936 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e779878-a547-4c02-8b0b-3d1facce350f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:49.195171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e779878-a547-4c02-8b0b-3d1facce350f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:49.196651 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e779878-a547-4c02-8b0b-3d1facce350f service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:49.206047 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1355/2736] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:34:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:34:49.454012 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309dbe8-3ff1-400e-bbe2-4602a5d4b6b1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:34:49.463720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309dbe8-3ff1-400e-bbe2-4602a5d4b6b1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:34:49.467360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e40fb-9994-4a49-81a9-41e8875eefc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:34:49.467997 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e40fb-9994-4a49-81a9-41e8875eefc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:34:49.468281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e40fb-9994-4a49-81a9-41e8875eefc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:34:49.468576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e40fb-9994-4a49-81a9-41e8875eefc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:34:49.473839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:34:49.527496 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e40fb-9994-4a49-81a9-41e8875eefc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:34:49.528915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e40fb-9994-4a49-81a9-41e8875eefc4 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:34:49.537792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1382/2737] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:34:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:34:59.201868 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=42,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:34:59.645349 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=24 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:36.436755 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e779878-a547-4c02-8b0b-3d1facce350f service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:36.498614 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e779878-a547-4c02-8b0b-3d1facce350f service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:36.503176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:36.503792 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:36.504068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:36.504334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] PATH_INFO: `/v3/users/eaa89695c37f40adb935978faceb662d` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:36.505451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa89695c37f40adb935978faceb662d)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:36.507363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:36.673408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa89695c37f40adb935978faceb662d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:36.685144 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa89695c37f40adb935978faceb662d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:36.691922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa89695c37f40adb935978faceb662d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:36.692591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa89695c37f40adb935978faceb662d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:36.693189 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa89695c37f40adb935978faceb662d'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:36.774008 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1356/2738] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:35:36 2022] DELETE /identity/v3/users/eaa89695c37f40adb935978faceb662d => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:36.779388 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab2a1-b029-461a-bcac-36ebdbfe0044 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:36.779902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab2a1-b029-461a-bcac-36ebdbfe0044 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:36.779902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab2a1-b029-461a-bcac-36ebdbfe0044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:36.799156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2e0ab2a1-b029-461a-bcac-36ebdbfe0044 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:36.800351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2e0ab2a1-b029-461a-bcac-36ebdbfe0044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:36.860571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1383/2739] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:35:36.870528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:36.871978 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:36.928402 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce302b2c-90ca-4d84-9d48-66a9cf4faad6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:36.932033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76974102-29fb-4318-a7c7-4054176d8ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:36.932622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76974102-29fb-4318-a7c7-4054176d8ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:36.932902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76974102-29fb-4318-a7c7-4054176d8ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:36.933176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76974102-29fb-4318-a7c7-4054176d8ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:36.941960 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76974102-29fb-4318-a7c7-4054176d8ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:36.943444 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76974102-29fb-4318-a7c7-4054176d8ed8 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:36.953324 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1357/2740] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:35:37.360031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ab2a1-b029-461a-bcac-36ebdbfe0044 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:37.361508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2e0ab2a1-b029-461a-bcac-36ebdbfe0044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:37.374342 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ab2a1-b029-461a-bcac-36ebdbfe0044 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:37.378941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:37.379593 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:37.379896 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:37.380171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] PATH_INFO: `/v3/projects/33d779180933436ca676e19c0ec8f35c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:37.381292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d779180933436ca676e19c0ec8f35c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:37.383243 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:37.472743 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d779180933436ca676e19c0ec8f35c', 'request_id': 'req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.523631 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d779180933436ca676e19c0ec8f35c', 'request_id': 'req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.529423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d779180933436ca676e19c0ec8f35c', 'request_id': 'req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.540316 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] The token cache is being invalidate because project 33d779180933436ca676e19c0ec8f35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:35:37.540604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.541270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.542737 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1384/2741] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:35:37 2022] DELETE /identity/v3/projects/33d779180933436ca676e19c0ec8f35c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:37.546744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76974102-29fb-4318-a7c7-4054176d8ed8 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:37.557002 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76974102-29fb-4318-a7c7-4054176d8ed8 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:37.575531 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:37.614119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:37.615105 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:37.619625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:37.619625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] PATH_INFO: `/v3/users/1ffa30cd80ad45918edbff5d466d7c3b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:37.619625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ffa30cd80ad45918edbff5d466d7c3b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:37.619937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:37.822312 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ffa30cd80ad45918edbff5d466d7c3b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.834096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ffa30cd80ad45918edbff5d466d7c3b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.842028 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ffa30cd80ad45918edbff5d466d7c3b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.842701 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ffa30cd80ad45918edbff5d466d7c3b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.843323 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-69c769ac-3e60-42da-8e6a-fca874ba854a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ffa30cd80ad45918edbff5d466d7c3b'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:37.936084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1358/2742] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:35:37 2022] DELETE /identity/v3/users/1ffa30cd80ad45918edbff5d466d7c3b => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:35:38.354907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:38.356427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:38.369449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a3eac-7c3e-4c86-a860-fe54fcbd5a0a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:38.426202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:38.426823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:38.427140 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:38.427423 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] PATH_INFO: `/v3/projects/e9f9f426cbeb4b5aa2d58eee35f7e54e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:38.428500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f9f426cbeb4b5aa2d58eee35f7e54e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:38.430362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:38.514172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:38.531858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f9f426cbeb4b5aa2d58eee35f7e54e', 'request_id': 'req-d1b3d684-0494-4fc1-b686-231bdbd5dc53'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:38.592380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f9f426cbeb4b5aa2d58eee35f7e54e', 'request_id': 'req-d1b3d684-0494-4fc1-b686-231bdbd5dc53'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:38.597934 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f9f426cbeb4b5aa2d58eee35f7e54e', 'request_id': 'req-d1b3d684-0494-4fc1-b686-231bdbd5dc53'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:38.609102 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] The token cache is being invalidate because project e9f9f426cbeb4b5aa2d58eee35f7e54e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:35:38.610950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:38.610950 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d1b3d684-0494-4fc1-b686-231bdbd5dc53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:38.612079 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1385/2743] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:35:38 2022] DELETE /identity/v3/projects/e9f9f426cbeb4b5aa2d58eee35f7e54e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:38.617006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711282a-6258-4a78-bf5a-76c51e5e3d7f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:38.617469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711282a-6258-4a78-bf5a-76c51e5e3d7f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:38.617469 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711282a-6258-4a78-bf5a-76c51e5e3d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:38.637405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-6711282a-6258-4a78-bf5a-76c51e5e3d7f None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:38.639687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-6711282a-6258-4a78-bf5a-76c51e5e3d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:38.695925 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1359/2744] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:35:38.704417 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f020-a9ab-46d8-b8d0-f2bacbf4e208 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:38.704705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f020-a9ab-46d8-b8d0-f2bacbf4e208 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:38.704983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f020-a9ab-46d8-b8d0-f2bacbf4e208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:38.724735 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8db2f020-a9ab-46d8-b8d0-f2bacbf4e208 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:38.725936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8db2f020-a9ab-46d8-b8d0-f2bacbf4e208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:38.733891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1386/2745] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:35:38.737999 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711282a-6258-4a78-bf5a-76c51e5e3d7f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:38.747649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711282a-6258-4a78-bf5a-76c51e5e3d7f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:38.752204 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:38.752791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:38.753071 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:38.753342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:38.754006 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:35:38.754306 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:38.756027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:38.768314 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1360/2746] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:35:38.772473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2f020-a9ab-46d8-b8d0-f2bacbf4e208 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:38.782322 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2f020-a9ab-46d8-b8d0-f2bacbf4e208 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:38.787048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a20fa-e2d2-472d-88b5-5d2b89178ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:38.787643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a20fa-e2d2-472d-88b5-5d2b89178ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:38.787928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a20fa-e2d2-472d-88b5-5d2b89178ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:38.788202 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a20fa-e2d2-472d-88b5-5d2b89178ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:38.788828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a20fa-e2d2-472d-88b5-5d2b89178ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:38.790715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a20fa-e2d2-472d-88b5-5d2b89178ad6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:38.867936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1387/2747] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:35:38 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:35:38.872748 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:38.882767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e9873-3a65-4430-bca0-15b6d0c83330 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:38.908940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:38.938452 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e989-2ab6-4dd0-8b2c-e0efb7ccdbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:38.939072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e989-2ab6-4dd0-8b2c-e0efb7ccdbce admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:38.939355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e989-2ab6-4dd0-8b2c-e0efb7ccdbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:38.939629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e989-2ab6-4dd0-8b2c-e0efb7ccdbce admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:38.940246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9e989-2ab6-4dd0-8b2c-e0efb7ccdbce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:38.942177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9e989-2ab6-4dd0-8b2c-e0efb7ccdbce admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.021152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1361/2748] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:35:38 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:35:39.025576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a20fa-e2d2-472d-88b5-5d2b89178ad6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.035924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a20fa-e2d2-472d-88b5-5d2b89178ad6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.041046 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e323cb9-f837-41a8-9b97-524187894747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.041733 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323cb9-f837-41a8-9b97-524187894747 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.042010 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323cb9-f837-41a8-9b97-524187894747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.042374 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323cb9-f837-41a8-9b97-524187894747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.042924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e323cb9-f837-41a8-9b97-524187894747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.044595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e323cb9-f837-41a8-9b97-524187894747 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.055262 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1388/2749] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:35:39.059938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e989-2ab6-4dd0-8b2c-e0efb7ccdbce admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.069897 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e989-2ab6-4dd0-8b2c-e0efb7ccdbce admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.075504 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42202c30-d5f1-4241-a86c-c7d3bbf4d679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.076093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42202c30-d5f1-4241-a86c-c7d3bbf4d679 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.076412 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42202c30-d5f1-4241-a86c-c7d3bbf4d679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.076679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42202c30-d5f1-4241-a86c-c7d3bbf4d679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.077315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42202c30-d5f1-4241-a86c-c7d3bbf4d679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.078923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42202c30-d5f1-4241-a86c-c7d3bbf4d679 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.089708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1362/2750] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:35:39.093776 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e323cb9-f837-41a8-9b97-524187894747 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.103850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e323cb9-f837-41a8-9b97-524187894747 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.108619 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562890cc-41eb-46b6-980c-30ab16a9e746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.109398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562890cc-41eb-46b6-980c-30ab16a9e746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.109785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562890cc-41eb-46b6-980c-30ab16a9e746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.110113 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562890cc-41eb-46b6-980c-30ab16a9e746 admin admin] PATH_INFO: `/v3/projects/c14234f1d451497082f7ded879b4902d/users/ffcedfd0f1f74d8c957f468470e8711e/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.127267 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562890cc-41eb-46b6-980c-30ab16a9e746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14234f1d451497082f7ded879b4902d, user_id=ffcedfd0f1f74d8c957f468470e8711e, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.130004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562890cc-41eb-46b6-980c-30ab16a9e746 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.210272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1389/2751] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:35:39 2022] PUT /identity/v3/projects/c14234f1d451497082f7ded879b4902d/users/ffcedfd0f1f74d8c957f468470e8711e/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:39.221944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22768dc-ce9d-4ef9-a0a5-89910003d5cd None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.222232 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22768dc-ce9d-4ef9-a0a5-89910003d5cd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.222506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22768dc-ce9d-4ef9-a0a5-89910003d5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.250681 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-e22768dc-ce9d-4ef9-a0a5-89910003d5cd None None] MFA Rules not processed for user `ffcedfd0f1f74d8c957f468470e8711e`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:39.252023 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e22768dc-ce9d-4ef9-a0a5-89910003d5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:39.317169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1363/2752] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:35:39.321725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562890cc-41eb-46b6-980c-30ab16a9e746 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.385530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562890cc-41eb-46b6-980c-30ab16a9e746 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.390106 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bf88b-a341-47ca-a893-40d60b27f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.390700 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bf88b-a341-47ca-a893-40d60b27f85e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.391318 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bf88b-a341-47ca-a893-40d60b27f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.391588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bf88b-a341-47ca-a893-40d60b27f85e admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.392214 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bf88b-a341-47ca-a893-40d60b27f85e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.394219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bf88b-a341-47ca-a893-40d60b27f85e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.481362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1390/2753] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:35:39 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:35:39.482934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22768dc-ce9d-4ef9-a0a5-89910003d5cd None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.493579 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22768dc-ce9d-4ef9-a0a5-89910003d5cd None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.550104 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d9044-7a74-4289-a95c-8fa850e28d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.550705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d9044-7a74-4289-a95c-8fa850e28d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.550983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d9044-7a74-4289-a95c-8fa850e28d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.551281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d9044-7a74-4289-a95c-8fa850e28d19 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.551892 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d9044-7a74-4289-a95c-8fa850e28d19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.553786 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d9044-7a74-4289-a95c-8fa850e28d19 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.656300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1364/2754] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:35:39 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:35:39.660304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bf88b-a341-47ca-a893-40d60b27f85e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.671353 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bf88b-a341-47ca-a893-40d60b27f85e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.675885 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6da6d5-99fe-4e74-973b-fb248cf038c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.676531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6da6d5-99fe-4e74-973b-fb248cf038c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.676806 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6da6d5-99fe-4e74-973b-fb248cf038c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.677081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6da6d5-99fe-4e74-973b-fb248cf038c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.677715 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6da6d5-99fe-4e74-973b-fb248cf038c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.679324 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6da6d5-99fe-4e74-973b-fb248cf038c5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.689847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1391/2755] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:35:39.693809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d9044-7a74-4289-a95c-8fa850e28d19 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.704091 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d9044-7a74-4289-a95c-8fa850e28d19 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.708940 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a791dc-9eb4-4774-8016-7092ad8da2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.709592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a791dc-9eb4-4774-8016-7092ad8da2f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.709872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a791dc-9eb4-4774-8016-7092ad8da2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.710146 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a791dc-9eb4-4774-8016-7092ad8da2f6 admin admin] PATH_INFO: `/v3/projects/d4ca66aafdf84c0d887be8cbb1c2dfde/users/20535dfa171c4135b33fc9539f19bfe5/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.725152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a791dc-9eb4-4774-8016-7092ad8da2f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ca66aafdf84c0d887be8cbb1c2dfde, user_id=20535dfa171c4135b33fc9539f19bfe5, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.727824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a791dc-9eb4-4774-8016-7092ad8da2f6 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.826407 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1365/2756] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:35:39 2022] PUT /identity/v3/projects/d4ca66aafdf84c0d887be8cbb1c2dfde/users/20535dfa171c4135b33fc9539f19bfe5/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:39.830351 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6da6d5-99fe-4e74-973b-fb248cf038c5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.841158 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6da6d5-99fe-4e74-973b-fb248cf038c5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.861111 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:39.896668 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a6a6a-870a-4358-819c-e0c17d1a43a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.897281 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a6a6a-870a-4358-819c-e0c17d1a43a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.897557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a6a6a-870a-4358-819c-e0c17d1a43a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.897826 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a6a6a-870a-4358-819c-e0c17d1a43a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.898454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a6a6a-870a-4358-819c-e0c17d1a43a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.900084 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a6a6a-870a-4358-819c-e0c17d1a43a4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.910303 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1392/2757] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:35:39.914413 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a791dc-9eb4-4774-8016-7092ad8da2f6 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:39.920555 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:39.925776 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a791dc-9eb4-4774-8016-7092ad8da2f6 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:39.930677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:39.931170 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.931458 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.931967 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] PATH_INFO: `/v3/projects/d4ca66aafdf84c0d887be8cbb1c2dfde/users/20535dfa171c4135b33fc9539f19bfe5/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.934048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ca66aafdf84c0d887be8cbb1c2dfde, user_id=20535dfa171c4135b33fc9539f19bfe5, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:39.936872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:39.946894 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1366/2758] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:35:39 2022] PUT /identity/v3/projects/d4ca66aafdf84c0d887be8cbb1c2dfde/users/20535dfa171c4135b33fc9539f19bfe5/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:39.957638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c12755-d660-496b-a859-fe19c3eb5a96 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:39.957924 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c12755-d660-496b-a859-fe19c3eb5a96 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:39.958204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c12755-d660-496b-a859-fe19c3eb5a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:39.986341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-56c12755-d660-496b-a859-fe19c3eb5a96 None None] MFA Rules not processed for user `20535dfa171c4135b33fc9539f19bfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:39.987632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-56c12755-d660-496b-a859-fe19c3eb5a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:40.054493 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1393/2759] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:35:40.066599 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:40.068279 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:40.126019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff2bee-0ce7-4f78-941e-c5107e568070 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:40.129574 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c921b-8877-4b4b-b132-4239c13f4bcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:40.130169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c921b-8877-4b4b-b132-4239c13f4bcd service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:40.130446 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c921b-8877-4b4b-b132-4239c13f4bcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:40.130717 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c921b-8877-4b4b-b132-4239c13f4bcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:40.139791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c921b-8877-4b4b-b132-4239c13f4bcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:40.141466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c921b-8877-4b4b-b132-4239c13f4bcd service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:40.150709 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1367/2760] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:35:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:35:40.359549 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c12755-d660-496b-a859-fe19c3eb5a96 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:40.369515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c12755-d660-496b-a859-fe19c3eb5a96 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:40.373244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb3f75-b615-4507-82c9-c9089f4e6390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:40.373850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3f75-b615-4507-82c9-c9089f4e6390 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:40.374134 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3f75-b615-4507-82c9-c9089f4e6390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:40.374411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3f75-b615-4507-82c9-c9089f4e6390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:40.430971 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb3f75-b615-4507-82c9-c9089f4e6390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:40.432902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb3f75-b615-4507-82c9-c9089f4e6390 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:40.441596 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1394/2761] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:35:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:35:50.146566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:50.437610 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=80 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:50.966677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c921b-8877-4b4b-b132-4239c13f4bcd service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:51.027589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c921b-8877-4b4b-b132-4239c13f4bcd service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:51.032687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:51.033300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:51.033575 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:51.033844 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] PATH_INFO: `/v3/users/ffcedfd0f1f74d8c957f468470e8711e` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:51.034937 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffcedfd0f1f74d8c957f468470e8711e)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:51.036855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:51.194162 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffcedfd0f1f74d8c957f468470e8711e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:51.206656 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffcedfd0f1f74d8c957f468470e8711e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:51.213172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffcedfd0f1f74d8c957f468470e8711e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:51.213845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffcedfd0f1f74d8c957f468470e8711e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:51.214451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffcedfd0f1f74d8c957f468470e8711e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:51.276944 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1368/2762] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:35:50 2022] DELETE /identity/v3/users/ffcedfd0f1f74d8c957f468470e8711e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:51.282042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c7860-a67c-4585-99b7-d0f1478b8410 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:51.282323 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c7860-a67c-4585-99b7-d0f1478b8410 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:51.282595 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c7860-a67c-4585-99b7-d0f1478b8410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:51.301176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b39c7860-a67c-4585-99b7-d0f1478b8410 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:51.302365 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b39c7860-a67c-4585-99b7-d0f1478b8410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:51.359137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1395/2763] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:35:51.368646 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:51.370018 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:51.427092 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d7417-6f1f-4ae3-9326-a0a56f75e528 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:51.430678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdaa04-e4bf-47dc-85d8-48cf19fb89df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:51.431311 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdaa04-e4bf-47dc-85d8-48cf19fb89df service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:51.431592 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdaa04-e4bf-47dc-85d8-48cf19fb89df service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:51.431860 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdaa04-e4bf-47dc-85d8-48cf19fb89df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:51.440661 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdaa04-e4bf-47dc-85d8-48cf19fb89df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:51.442084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdaa04-e4bf-47dc-85d8-48cf19fb89df service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:51.452081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1369/2764] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:35:51.857348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c7860-a67c-4585-99b7-d0f1478b8410 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:51.861341 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b39c7860-a67c-4585-99b7-d0f1478b8410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:51.870699 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c7860-a67c-4585-99b7-d0f1478b8410 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:51.877665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:51.878328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:51.878611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:51.878889 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] PATH_INFO: `/v3/projects/c14234f1d451497082f7ded879b4902d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:51.880099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c14234f1d451497082f7ded879b4902d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:51.882026 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:51.983053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c14234f1d451497082f7ded879b4902d', 'request_id': 'req-85faf53a-d228-4a11-bfdc-d30ba73c8781'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.054761 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c14234f1d451497082f7ded879b4902d', 'request_id': 'req-85faf53a-d228-4a11-bfdc-d30ba73c8781'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.061215 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c14234f1d451497082f7ded879b4902d', 'request_id': 'req-85faf53a-d228-4a11-bfdc-d30ba73c8781'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.072362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] The token cache is being invalidate because project c14234f1d451497082f7ded879b4902d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:35:52.072653 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.073338 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.074917 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1396/2765] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:35:51 2022] DELETE /identity/v3/projects/c14234f1d451497082f7ded879b4902d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:52.078877 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdaa04-e4bf-47dc-85d8-48cf19fb89df service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:52.088730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdaa04-e4bf-47dc-85d8-48cf19fb89df service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:52.106705 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:52.146264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:52.146893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:52.147200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:52.147473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] PATH_INFO: `/v3/users/20535dfa171c4135b33fc9539f19bfe5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:52.148584 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20535dfa171c4135b33fc9539f19bfe5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:52.150490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:52.348791 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20535dfa171c4135b33fc9539f19bfe5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.361121 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20535dfa171c4135b33fc9539f19bfe5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.367839 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20535dfa171c4135b33fc9539f19bfe5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.368510 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20535dfa171c4135b33fc9539f19bfe5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.369109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2bbaf40d-86cf-4561-ac1e-1e9bc84742d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20535dfa171c4135b33fc9539f19bfe5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:52.453700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1370/2766] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:35:52 2022] DELETE /identity/v3/users/20535dfa171c4135b33fc9539f19bfe5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:52.850966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:52.852402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:52.865946 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85faf53a-d228-4a11-bfdc-d30ba73c8781 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:52.925543 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:52.926172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:52.926454 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:52.926725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] PATH_INFO: `/v3/projects/d4ca66aafdf84c0d887be8cbb1c2dfde` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:52.927862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ca66aafdf84c0d887be8cbb1c2dfde)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:52.929752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:53.016788 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:53.032725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ca66aafdf84c0d887be8cbb1c2dfde', 'request_id': 'req-08f77495-3874-47ac-b729-b3dd3214b926'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:53.106891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ca66aafdf84c0d887be8cbb1c2dfde', 'request_id': 'req-08f77495-3874-47ac-b729-b3dd3214b926'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:53.113331 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ca66aafdf84c0d887be8cbb1c2dfde', 'request_id': 'req-08f77495-3874-47ac-b729-b3dd3214b926'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:53.125505 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] The token cache is being invalidate because project d4ca66aafdf84c0d887be8cbb1c2dfde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:35:53.125796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:53.126469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-08f77495-3874-47ac-b729-b3dd3214b926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:35:53.128075 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1397/2767] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:35:52 2022] DELETE /identity/v3/projects/d4ca66aafdf84c0d887be8cbb1c2dfde => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:53.132832 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eb09-f7a8-4cee-8653-2e0674eaa73c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.133118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eb09-f7a8-4cee-8653-2e0674eaa73c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.133418 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eb09-f7a8-4cee-8653-2e0674eaa73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.154569 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-f212eb09-f7a8-4cee-8653-2e0674eaa73c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:53.155807 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f212eb09-f7a8-4cee-8653-2e0674eaa73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:53.213830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1371/2768] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:35:53.223272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2131a3-e8c1-4bf2-bc19-9e31753854b6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.223557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2131a3-e8c1-4bf2-bc19-9e31753854b6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.223833 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2131a3-e8c1-4bf2-bc19-9e31753854b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.243469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1a2131a3-e8c1-4bf2-bc19-9e31753854b6 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:53.244673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1a2131a3-e8c1-4bf2-bc19-9e31753854b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:53.252940 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1398/2769] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:35:53.257054 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212eb09-f7a8-4cee-8653-2e0674eaa73c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:53.267365 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212eb09-f7a8-4cee-8653-2e0674eaa73c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:53.271986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:53.272578 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.272856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.273129 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.273767 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:35:53.274067 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:53.275726 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:53.287260 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1372/2770] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:35:53.291449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2131a3-e8c1-4bf2-bc19-9e31753854b6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:53.300669 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2131a3-e8c1-4bf2-bc19-9e31753854b6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:53.305283 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a930654-eade-426a-a834-69f900fa7a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:53.305864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a930654-eade-426a-a834-69f900fa7a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.306137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a930654-eade-426a-a834-69f900fa7a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.306402 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a930654-eade-426a-a834-69f900fa7a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.307039 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a930654-eade-426a-a834-69f900fa7a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:53.309478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a930654-eade-426a-a834-69f900fa7a5b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:53.422665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1399/2771] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:35:53 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:35:53.426783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:53.437520 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a480-c83f-4424-b30d-8d243ee4c18c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:53.462157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:53.492082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3943c-c208-4025-a1c9-94486c291567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:53.492676 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3943c-c208-4025-a1c9-94486c291567 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.492952 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3943c-c208-4025-a1c9-94486c291567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.493221 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3943c-c208-4025-a1c9-94486c291567 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.493829 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3943c-c208-4025-a1c9-94486c291567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:53.495812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3943c-c208-4025-a1c9-94486c291567 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:53.586637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1373/2772] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:35:53 2022] POST /identity/v3/users => generated 282 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:35:53.590817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a930654-eade-426a-a834-69f900fa7a5b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:53.601157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a930654-eade-426a-a834-69f900fa7a5b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:53.606613 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dceb1-c25d-4651-9945-d72e5c80ec28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:53.607237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dceb1-c25d-4651-9945-d72e5c80ec28 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.607514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dceb1-c25d-4651-9945-d72e5c80ec28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.607787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dceb1-c25d-4651-9945-d72e5c80ec28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.608506 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dceb1-c25d-4651-9945-d72e5c80ec28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:53.610123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dceb1-c25d-4651-9945-d72e5c80ec28 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:53.620119 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1400/2773] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:35:53.624738 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3943c-c208-4025-a1c9-94486c291567 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:53.634677 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3943c-c208-4025-a1c9-94486c291567 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:53.639387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f0d24-69d8-4ca3-b6bf-de7251f5af67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:53.639972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0d24-69d8-4ca3-b6bf-de7251f5af67 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.640251 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0d24-69d8-4ca3-b6bf-de7251f5af67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.640525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0d24-69d8-4ca3-b6bf-de7251f5af67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.641417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f0d24-69d8-4ca3-b6bf-de7251f5af67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:53.643019 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f0d24-69d8-4ca3-b6bf-de7251f5af67 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:53.653395 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1374/2774] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:35:53.657394 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dceb1-c25d-4651-9945-d72e5c80ec28 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:53.667309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dceb1-c25d-4651-9945-d72e5c80ec28 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:53.672536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0af8a-4d44-4f4b-afb9-a74ca8bb0209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:53.673221 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0af8a-4d44-4f4b-afb9-a74ca8bb0209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.673448 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0af8a-4d44-4f4b-afb9-a74ca8bb0209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.673724 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0af8a-4d44-4f4b-afb9-a74ca8bb0209 admin admin] PATH_INFO: `/v3/projects/adfa196b03794619a649ecf20afa93e3/users/26d6e8ae57d147739be9228008409f8a/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.688949 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0af8a-4d44-4f4b-afb9-a74ca8bb0209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adfa196b03794619a649ecf20afa93e3, user_id=26d6e8ae57d147739be9228008409f8a, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:53.691612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0af8a-4d44-4f4b-afb9-a74ca8bb0209 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:53.764302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1401/2775] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:35:53 2022] PUT /identity/v3/projects/adfa196b03794619a649ecf20afa93e3/users/26d6e8ae57d147739be9228008409f8a/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:35:53.831708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b875586-6b40-479a-950d-66733f611e1c None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.831988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b875586-6b40-479a-950d-66733f611e1c None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.832255 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b875586-6b40-479a-950d-66733f611e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.860354 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3b875586-6b40-479a-950d-66733f611e1c None None] MFA Rules not processed for user `26d6e8ae57d147739be9228008409f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:53.861550 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3b875586-6b40-479a-950d-66733f611e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:53.926853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1375/2776] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:35:53.931358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0af8a-4d44-4f4b-afb9-a74ca8bb0209 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:53.992272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0af8a-4d44-4f4b-afb9-a74ca8bb0209 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:53.996935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78373fe8-e505-47eb-ad95-6f122b6874ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:53.997540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373fe8-e505-47eb-ad95-6f122b6874ae admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:53.997823 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373fe8-e505-47eb-ad95-6f122b6874ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:53.998099 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373fe8-e505-47eb-ad95-6f122b6874ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:53.998727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78373fe8-e505-47eb-ad95-6f122b6874ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:54.000660 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78373fe8-e505-47eb-ad95-6f122b6874ae admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:54.084335 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1402/2777] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:35:53 2022] POST /identity/v3/projects => generated 361 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:35:54.088281 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b875586-6b40-479a-950d-66733f611e1c None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:54.099547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b875586-6b40-479a-950d-66733f611e1c None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:54.156830 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4da7d-5c10-4009-b432-f147eb44a92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:54.157442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4da7d-5c10-4009-b432-f147eb44a92e admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:54.157785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4da7d-5c10-4009-b432-f147eb44a92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:54.158064 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4da7d-5c10-4009-b432-f147eb44a92e admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:54.158696 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4da7d-5c10-4009-b432-f147eb44a92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:54.160695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4da7d-5c10-4009-b432-f147eb44a92e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:54.260405 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1376/2778] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:35:54 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:35:54.265128 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78373fe8-e505-47eb-ad95-6f122b6874ae admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:54.275665 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78373fe8-e505-47eb-ad95-6f122b6874ae admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:54.281387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18befd3-b6f7-4d7a-a5fd-f8588852486e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:54.281982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18befd3-b6f7-4d7a-a5fd-f8588852486e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:54.282264 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18befd3-b6f7-4d7a-a5fd-f8588852486e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:54.282537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18befd3-b6f7-4d7a-a5fd-f8588852486e admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:54.283209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18befd3-b6f7-4d7a-a5fd-f8588852486e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:54.284794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18befd3-b6f7-4d7a-a5fd-f8588852486e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:54.295502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1403/2779] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:35:54.299501 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4da7d-5c10-4009-b432-f147eb44a92e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:54.309506 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4da7d-5c10-4009-b432-f147eb44a92e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:54.314029 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1609552-478a-433e-bcda-06cd19230a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:54.314699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1609552-478a-433e-bcda-06cd19230a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:54.314980 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1609552-478a-433e-bcda-06cd19230a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:54.315287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1609552-478a-433e-bcda-06cd19230a4a admin admin] PATH_INFO: `/v3/projects/f6f55e1347574d85aa8ba977140684f2/users/9a70ce0805604dc189e0c0f844e3235e/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:54.331521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1609552-478a-433e-bcda-06cd19230a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6f55e1347574d85aa8ba977140684f2, user_id=9a70ce0805604dc189e0c0f844e3235e, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:54.334120 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1609552-478a-433e-bcda-06cd19230a4a admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:54.380090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1377/2780] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:35:54 2022] PUT /identity/v3/projects/f6f55e1347574d85aa8ba977140684f2/users/9a70ce0805604dc189e0c0f844e3235e/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:54.383955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18befd3-b6f7-4d7a-a5fd-f8588852486e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:54.394836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18befd3-b6f7-4d7a-a5fd-f8588852486e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:54.414781 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:54.451208 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f822cf-245c-4851-920d-7338a0b7d540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:54.451813 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f822cf-245c-4851-920d-7338a0b7d540 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:54.452116 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f822cf-245c-4851-920d-7338a0b7d540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:54.452391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f822cf-245c-4851-920d-7338a0b7d540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:54.453030 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f822cf-245c-4851-920d-7338a0b7d540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:54.454622 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f822cf-245c-4851-920d-7338a0b7d540 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:54.464723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1404/2781] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:35:54.469658 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1609552-478a-433e-bcda-06cd19230a4a admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:54.475206 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:35:54.480496 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1609552-478a-433e-bcda-06cd19230a4a admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:54.485361 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:54.486000 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:54.486277 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:54.486544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] PATH_INFO: `/v3/projects/f6f55e1347574d85aa8ba977140684f2/users/9a70ce0805604dc189e0c0f844e3235e/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:54.488567 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6f55e1347574d85aa8ba977140684f2, user_id=9a70ce0805604dc189e0c0f844e3235e, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:54.491294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:54.502850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1378/2782] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:35:54 2022] PUT /identity/v3/projects/f6f55e1347574d85aa8ba977140684f2/users/9a70ce0805604dc189e0c0f844e3235e/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:35:54.513566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f1e3f-445b-4411-ad0c-f9fc3a9bdb0f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:54.513777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f1e3f-445b-4411-ad0c-f9fc3a9bdb0f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:54.514049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f1e3f-445b-4411-ad0c-f9fc3a9bdb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:54.542452 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-028f1e3f-445b-4411-ad0c-f9fc3a9bdb0f None None] MFA Rules not processed for user `9a70ce0805604dc189e0c0f844e3235e`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:35:54.543861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-028f1e3f-445b-4411-ad0c-f9fc3a9bdb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:54.611977 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1405/2783] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:35:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:35:54.624294 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:54.625979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:35:54.682635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680dc01-028a-4b6c-a59a-0bc47f730a38 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:54.687049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9a9eb-21f6-47cc-a58b-1844e4fad904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:54.687643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a9eb-21f6-47cc-a58b-1844e4fad904 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:54.687922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a9eb-21f6-47cc-a58b-1844e4fad904 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:54.688194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9a9eb-21f6-47cc-a58b-1844e4fad904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:54.696938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9a9eb-21f6-47cc-a58b-1844e4fad904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:54.698359 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9a9eb-21f6-47cc-a58b-1844e4fad904 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:54.708489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1379/2784] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:35:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:35:54.959694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f1e3f-445b-4411-ad0c-f9fc3a9bdb0f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:35:54.969519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f1e3f-445b-4411-ad0c-f9fc3a9bdb0f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:35:54.972985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e3f34-b7da-48a4-aac2-6f891ca5074b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:35:54.973570 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3f34-b7da-48a4-aac2-6f891ca5074b service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:35:54.973890 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3f34-b7da-48a4-aac2-6f891ca5074b service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:35:54.974160 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3f34-b7da-48a4-aac2-6f891ca5074b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:35:55.029319 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e3f34-b7da-48a4-aac2-6f891ca5074b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:35:55.030730 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e3f34-b7da-48a4-aac2-6f891ca5074b service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:35:55.039348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1406/2785] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:35:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:04.704448 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:05.035519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=80 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:05.759856 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9a9eb-21f6-47cc-a58b-1844e4fad904 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:05.823643 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9a9eb-21f6-47cc-a58b-1844e4fad904 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:05.828152 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:05.828773 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:05.829048 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:05.829317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] PATH_INFO: `/v3/users/26d6e8ae57d147739be9228008409f8a` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:05.830468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=26d6e8ae57d147739be9228008409f8a)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:05.832400 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:06.011401 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26d6e8ae57d147739be9228008409f8a'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.024650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26d6e8ae57d147739be9228008409f8a'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.031919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26d6e8ae57d147739be9228008409f8a'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.032648 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26d6e8ae57d147739be9228008409f8a'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.033249 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26d6e8ae57d147739be9228008409f8a'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.129136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1380/2786] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:05 2022] DELETE /identity/v3/users/26d6e8ae57d147739be9228008409f8a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:06.133629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08415afd-92a6-4bfe-8097-46a82bcc5ebf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:06.139272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08415afd-92a6-4bfe-8097-46a82bcc5ebf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:06.139272 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08415afd-92a6-4bfe-8097-46a82bcc5ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:06.154603 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-08415afd-92a6-4bfe-8097-46a82bcc5ebf None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:06.155828 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-08415afd-92a6-4bfe-8097-46a82bcc5ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:06.214574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1407/2787] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:06.224332 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:06.225745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:06.282242 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4bf6f-05ac-4b7c-8afa-d763ad93e9bd admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:06.285813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f2b62-3c56-4d9d-8fd3-d401ea5b15cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:06.286404 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2b62-3c56-4d9d-8fd3-d401ea5b15cb service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:06.286679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2b62-3c56-4d9d-8fd3-d401ea5b15cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:06.286946 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2b62-3c56-4d9d-8fd3-d401ea5b15cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:06.295610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f2b62-3c56-4d9d-8fd3-d401ea5b15cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:06.297606 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f2b62-3c56-4d9d-8fd3-d401ea5b15cb service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:06.306764 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1381/2788] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:06.736982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08415afd-92a6-4bfe-8097-46a82bcc5ebf None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:06.738447 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-08415afd-92a6-4bfe-8097-46a82bcc5ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:06.750270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08415afd-92a6-4bfe-8097-46a82bcc5ebf None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:06.754995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:06.755654 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:06.755935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:06.756209 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] PATH_INFO: `/v3/projects/adfa196b03794619a649ecf20afa93e3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:06.757348 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adfa196b03794619a649ecf20afa93e3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:06.759276 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:06.888891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adfa196b03794619a649ecf20afa93e3', 'request_id': 'req-d6055f14-4715-4c73-adaa-1cf58f4da061'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.957752 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adfa196b03794619a649ecf20afa93e3', 'request_id': 'req-d6055f14-4715-4c73-adaa-1cf58f4da061'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.963792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adfa196b03794619a649ecf20afa93e3', 'request_id': 'req-d6055f14-4715-4c73-adaa-1cf58f4da061'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.975787 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] The token cache is being invalidate because project adfa196b03794619a649ecf20afa93e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:06.976078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.976710 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:06.978289 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1408/2789] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:06 2022] DELETE /identity/v3/projects/adfa196b03794619a649ecf20afa93e3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:06.982475 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f2b62-3c56-4d9d-8fd3-d401ea5b15cb service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:06.992188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f2b62-3c56-4d9d-8fd3-d401ea5b15cb service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:07.010022 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:07.049644 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:07.050264 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:07.050542 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:07.050812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] PATH_INFO: `/v3/users/9a70ce0805604dc189e0c0f844e3235e` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:07.051979 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a70ce0805604dc189e0c0f844e3235e)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:07.054125 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:07.251246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a70ce0805604dc189e0c0f844e3235e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:07.262730 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a70ce0805604dc189e0c0f844e3235e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:07.270521 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a70ce0805604dc189e0c0f844e3235e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:07.271257 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a70ce0805604dc189e0c0f844e3235e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:07.271962 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bab1fee8-10f2-4b6a-a73f-806d83f8304e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a70ce0805604dc189e0c0f844e3235e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:07.340095 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1382/2790] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:06 2022] DELETE /identity/v3/users/9a70ce0805604dc189e0c0f844e3235e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:07.765176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:07.766871 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:07.781091 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055f14-4715-4c73-adaa-1cf58f4da061 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:07.840307 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:07.840935 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:07.841217 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:07.841492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] PATH_INFO: `/v3/projects/f6f55e1347574d85aa8ba977140684f2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:07.842606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6f55e1347574d85aa8ba977140684f2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:07.845151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:07.930860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:07.947171 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6f55e1347574d85aa8ba977140684f2', 'request_id': 'req-c0a287dd-085b-4498-aa90-c23e4ee827b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:08.003291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6f55e1347574d85aa8ba977140684f2', 'request_id': 'req-c0a287dd-085b-4498-aa90-c23e4ee827b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:08.009031 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6f55e1347574d85aa8ba977140684f2', 'request_id': 'req-c0a287dd-085b-4498-aa90-c23e4ee827b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:08.025251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] The token cache is being invalidate because project f6f55e1347574d85aa8ba977140684f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:08.025251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:08.025251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c0a287dd-085b-4498-aa90-c23e4ee827b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:08.025251 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1409/2791] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:07 2022] DELETE /identity/v3/projects/f6f55e1347574d85aa8ba977140684f2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:08.028576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae520e-3d79-4579-a40f-0c0840c3e0ec None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.029075 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae520e-3d79-4579-a40f-0c0840c3e0ec None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.029193 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae520e-3d79-4579-a40f-0c0840c3e0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.049372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-85ae520e-3d79-4579-a40f-0c0840c3e0ec None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:08.050595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-85ae520e-3d79-4579-a40f-0c0840c3e0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:08.110084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1383/2792] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:08.119070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac03693-53a6-42eb-ae55-a271704e2dfd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.119360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac03693-53a6-42eb-ae55-a271704e2dfd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.119632 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac03693-53a6-42eb-ae55-a271704e2dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.138841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-eac03693-53a6-42eb-ae55-a271704e2dfd None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:08.140053 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eac03693-53a6-42eb-ae55-a271704e2dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:08.148147 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1410/2793] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:36:08.152398 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae520e-3d79-4579-a40f-0c0840c3e0ec None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.162813 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae520e-3d79-4579-a40f-0c0840c3e0ec None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.167540 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.168138 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.168417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.168689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.169331 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:36:08.169632 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.171261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:08.183678 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1384/2794] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:36:08.188002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac03693-53a6-42eb-ae55-a271704e2dfd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.197019 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac03693-53a6-42eb-ae55-a271704e2dfd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.201606 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1719f-927b-4698-b6ea-0ae4712514f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.202191 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1719f-927b-4698-b6ea-0ae4712514f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.202469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1719f-927b-4698-b6ea-0ae4712514f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.202738 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1719f-927b-4698-b6ea-0ae4712514f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.203434 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1719f-927b-4698-b6ea-0ae4712514f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.205309 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1719f-927b-4698-b6ea-0ae4712514f4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:08.285804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1411/2795] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:36:08 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:36:08.289742 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.299745 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305adc0-c3ea-4d63-8c9e-35b87aa3cf4f admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.324589 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:08.354945 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb688caf-971b-4833-ac61-92901084c1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.355566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688caf-971b-4833-ac61-92901084c1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.355841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688caf-971b-4833-ac61-92901084c1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.356111 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688caf-971b-4833-ac61-92901084c1f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.356719 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb688caf-971b-4833-ac61-92901084c1f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.358609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb688caf-971b-4833-ac61-92901084c1f5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:08.438214 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1385/2796] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:36:08 2022] POST /identity/v3/users => generated 282 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:36:08.442339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1719f-927b-4698-b6ea-0ae4712514f4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.452576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1719f-927b-4698-b6ea-0ae4712514f4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.458148 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8b2c8-ac1d-4e4c-9e9b-58a75b3d2019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.458860 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8b2c8-ac1d-4e4c-9e9b-58a75b3d2019 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.459212 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8b2c8-ac1d-4e4c-9e9b-58a75b3d2019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.459574 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8b2c8-ac1d-4e4c-9e9b-58a75b3d2019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.460226 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8b2c8-ac1d-4e4c-9e9b-58a75b3d2019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.461868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8b2c8-ac1d-4e4c-9e9b-58a75b3d2019 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:08.472609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1412/2797] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:08.476540 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb688caf-971b-4833-ac61-92901084c1f5 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.486333 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb688caf-971b-4833-ac61-92901084c1f5 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.490923 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.491546 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.491823 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.492096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.492731 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.494378 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:08.505340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1386/2798] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:08.509410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8b2c8-ac1d-4e4c-9e9b-58a75b3d2019 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.519839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8b2c8-ac1d-4e4c-9e9b-58a75b3d2019 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.524396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f27b9a-f389-4d1f-ba7a-335523a5f3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.525038 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27b9a-f389-4d1f-ba7a-335523a5f3a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.525297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27b9a-f389-4d1f-ba7a-335523a5f3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.525567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27b9a-f389-4d1f-ba7a-335523a5f3a0 admin admin] PATH_INFO: `/v3/projects/af58b364073445afbcf818c2e09ef4d7/users/0eee3ce01eda41639b0ded844ae0d8bd/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.540591 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f27b9a-f389-4d1f-ba7a-335523a5f3a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af58b364073445afbcf818c2e09ef4d7, user_id=0eee3ce01eda41639b0ded844ae0d8bd, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.543142 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f27b9a-f389-4d1f-ba7a-335523a5f3a0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:08.633169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1413/2799] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:08 2022] PUT /identity/v3/projects/af58b364073445afbcf818c2e09ef4d7/users/0eee3ce01eda41639b0ded844ae0d8bd/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:08.639169 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.642158 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:08.644049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dc6dc-d6f3-491a-b510-dcba41f89146 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.644325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dc6dc-d6f3-491a-b510-dcba41f89146 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.644604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dc6dc-d6f3-491a-b510-dcba41f89146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.672609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b02dc6dc-d6f3-491a-b510-dcba41f89146 None None] MFA Rules not processed for user `0eee3ce01eda41639b0ded844ae0d8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:08.673867 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b02dc6dc-d6f3-491a-b510-dcba41f89146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:08.714109 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf3f7-9963-4927-92f7-66dacd36d909 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.739839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1414/2800] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:36:08.744337 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dc6dc-d6f3-491a-b510-dcba41f89146 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.778354 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.779065 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.779334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.779605 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.781943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:08.787409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:08.798663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.798663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:08.806449 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1387/2801] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:36:08.808576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dc6dc-d6f3-491a-b510-dcba41f89146 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.814011 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd5e06-4737-4811-9aec-f7ad2aa5ad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.814729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5e06-4737-4811-9aec-f7ad2aa5ad46 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.815048 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5e06-4737-4811-9aec-f7ad2aa5ad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.815329 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5e06-4737-4811-9aec-f7ad2aa5ad46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.815957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd5e06-4737-4811-9aec-f7ad2aa5ad46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.817841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd5e06-4737-4811-9aec-f7ad2aa5ad46 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:08.890881 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1415/2802] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:36:08 2022] POST /identity/v3/projects => generated 361 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:36:08.895142 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:08.905389 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026e077-e924-4d06-b43c-35ea5c7ff15d service placement] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:08.943934 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:08.962081 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbe4cc-fbd3-41a6-b634-3d35f596edc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:08.962695 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbe4cc-fbd3-41a6-b634-3d35f596edc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:08.963212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbe4cc-fbd3-41a6-b634-3d35f596edc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:08.963725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbe4cc-fbd3-41a6-b634-3d35f596edc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:08.964388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbe4cc-fbd3-41a6-b634-3d35f596edc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:08.966853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbe4cc-fbd3-41a6-b634-3d35f596edc4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:09.044238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1388/2803] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:36:08 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:36:09.051132 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd5e06-4737-4811-9aec-f7ad2aa5ad46 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:09.060004 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd5e06-4737-4811-9aec-f7ad2aa5ad46 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:09.065346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac674a-8efd-4776-8349-a46386ad4ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:09.068379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac674a-8efd-4776-8349-a46386ad4ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:09.068379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac674a-8efd-4776-8349-a46386ad4ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:09.068379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac674a-8efd-4776-8349-a46386ad4ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:09.068379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac674a-8efd-4776-8349-a46386ad4ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:09.069339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac674a-8efd-4776-8349-a46386ad4ebf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:09.080003 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1416/2804] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:09.084282 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbe4cc-fbd3-41a6-b634-3d35f596edc4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:09.094168 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbe4cc-fbd3-41a6-b634-3d35f596edc4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:09.098722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f1076-b41c-4a96-b4e8-2272e5e6589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:09.099433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f1076-b41c-4a96-b4e8-2272e5e6589e admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:09.099675 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f1076-b41c-4a96-b4e8-2272e5e6589e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:09.099948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f1076-b41c-4a96-b4e8-2272e5e6589e admin admin] PATH_INFO: `/v3/projects/cfbd8b42779c48d79b991474f0352628/users/411c6c18366f487ab38980cf00e77af6/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:09.125816 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f1076-b41c-4a96-b4e8-2272e5e6589e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfbd8b42779c48d79b991474f0352628, user_id=411c6c18366f487ab38980cf00e77af6, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:09.128679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f1076-b41c-4a96-b4e8-2272e5e6589e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:09.186638 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1389/2805] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:09 2022] PUT /identity/v3/projects/cfbd8b42779c48d79b991474f0352628/users/411c6c18366f487ab38980cf00e77af6/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:09.192785 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac674a-8efd-4776-8349-a46386ad4ebf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:09.201582 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac674a-8efd-4776-8349-a46386ad4ebf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:09.256407 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb25795-e20b-418c-8d16-724825faef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:09.256994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb25795-e20b-418c-8d16-724825faef69 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:09.257270 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb25795-e20b-418c-8d16-724825faef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:09.257538 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb25795-e20b-418c-8d16-724825faef69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:09.258168 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb25795-e20b-418c-8d16-724825faef69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:09.259771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb25795-e20b-418c-8d16-724825faef69 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:09.269587 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1417/2806] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:09.273756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f1076-b41c-4a96-b4e8-2272e5e6589e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:09.284093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f1076-b41c-4a96-b4e8-2272e5e6589e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:09.288624 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:09.289265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:09.289543 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:09.289814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] PATH_INFO: `/v3/projects/cfbd8b42779c48d79b991474f0352628/users/411c6c18366f487ab38980cf00e77af6/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:09.291747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfbd8b42779c48d79b991474f0352628, user_id=411c6c18366f487ab38980cf00e77af6, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:09.294349 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:09.305005 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1390/2807] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:09 2022] PUT /identity/v3/projects/cfbd8b42779c48d79b991474f0352628/users/411c6c18366f487ab38980cf00e77af6/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:09.316429 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfaeaa-0b0b-4677-9ca2-d6422fed8598 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:09.316720 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfaeaa-0b0b-4677-9ca2-d6422fed8598 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:09.317000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfaeaa-0b0b-4677-9ca2-d6422fed8598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:09.345412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a6bfaeaa-0b0b-4677-9ca2-d6422fed8598 None None] MFA Rules not processed for user `411c6c18366f487ab38980cf00e77af6`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:09.346727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6bfaeaa-0b0b-4677-9ca2-d6422fed8598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:09.415519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1418/2808] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:09.436303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:09.438171 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:09.503822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871d0ec-dbbf-4a82-9308-82c872f8f03b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:09.507620 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6cc78-4d23-499e-b572-a6e0a56de36e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:09.508227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6cc78-4d23-499e-b572-a6e0a56de36e service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:09.508509 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6cc78-4d23-499e-b572-a6e0a56de36e service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:09.508784 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6cc78-4d23-499e-b572-a6e0a56de36e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:09.518238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6cc78-4d23-499e-b572-a6e0a56de36e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:09.519722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6cc78-4d23-499e-b572-a6e0a56de36e service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:09.529778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1391/2809] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:09.747904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfaeaa-0b0b-4677-9ca2-d6422fed8598 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:09.757967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfaeaa-0b0b-4677-9ca2-d6422fed8598 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:09.761794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7844b-bb53-4548-bc9c-729ff3b63864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:09.762396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7844b-bb53-4548-bc9c-729ff3b63864 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:09.762679 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7844b-bb53-4548-bc9c-729ff3b63864 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:09.762952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7844b-bb53-4548-bc9c-729ff3b63864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:09.796379 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:09.820392 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7844b-bb53-4548-bc9c-729ff3b63864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:09.823612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7844b-bb53-4548-bc9c-729ff3b63864 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:09.831404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1419/2810] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:13.128188 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6cc78-4d23-499e-b572-a6e0a56de36e service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:13.189112 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6cc78-4d23-499e-b572-a6e0a56de36e service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:13.193723 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:13.194339 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:13.194616 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:13.194890 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] PATH_INFO: `/v3/users/0eee3ce01eda41639b0ded844ae0d8bd` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:13.196026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eee3ce01eda41639b0ded844ae0d8bd)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:13.197922 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:13.356194 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:13.364814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eee3ce01eda41639b0ded844ae0d8bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:13.376470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eee3ce01eda41639b0ded844ae0d8bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:13.383247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eee3ce01eda41639b0ded844ae0d8bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:13.383915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eee3ce01eda41639b0ded844ae0d8bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:13.384554 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eee3ce01eda41639b0ded844ae0d8bd'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:13.436961 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1392/2811] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:13 2022] DELETE /identity/v3/users/0eee3ce01eda41639b0ded844ae0d8bd => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:13.441590 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516e3a-0239-4a86-ae91-ef09b1ddaf88 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:13.442137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516e3a-0239-4a86-ae91-ef09b1ddaf88 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:13.442137 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516e3a-0239-4a86-ae91-ef09b1ddaf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:13.461157 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-40516e3a-0239-4a86-ae91-ef09b1ddaf88 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:13.462519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-40516e3a-0239-4a86-ae91-ef09b1ddaf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:13.521650 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1420/2812] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:13.531363 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:13.532751 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:13.587878 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e150c-09d5-4bb4-8677-b3d4fdc4ab09 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:13.591460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa140a-b562-4541-a3ae-6fa50f309eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:13.592050 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa140a-b562-4541-a3ae-6fa50f309eee service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:13.592330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa140a-b562-4541-a3ae-6fa50f309eee service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:13.592597 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa140a-b562-4541-a3ae-6fa50f309eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:13.601884 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa140a-b562-4541-a3ae-6fa50f309eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:13.603344 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa140a-b562-4541-a3ae-6fa50f309eee service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:13.612470 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1393/2813] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:14.015816 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40516e3a-0239-4a86-ae91-ef09b1ddaf88 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:14.017525 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-40516e3a-0239-4a86-ae91-ef09b1ddaf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:14.031070 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40516e3a-0239-4a86-ae91-ef09b1ddaf88 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:14.035991 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:14.036677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:14.036957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:14.038034 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] PATH_INFO: `/v3/projects/af58b364073445afbcf818c2e09ef4d7` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:14.039456 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af58b364073445afbcf818c2e09ef4d7)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:14.041492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:14.169838 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af58b364073445afbcf818c2e09ef4d7', 'request_id': 'req-03d447fe-32f2-4d35-8d8c-24bff02163e4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.230723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af58b364073445afbcf818c2e09ef4d7', 'request_id': 'req-03d447fe-32f2-4d35-8d8c-24bff02163e4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.237041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af58b364073445afbcf818c2e09ef4d7', 'request_id': 'req-03d447fe-32f2-4d35-8d8c-24bff02163e4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.247961 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] The token cache is being invalidate because project af58b364073445afbcf818c2e09ef4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:14.248246 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.248863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.250321 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1421/2814] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:14 2022] DELETE /identity/v3/projects/af58b364073445afbcf818c2e09ef4d7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:14.255118 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa140a-b562-4541-a3ae-6fa50f309eee service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:14.265131 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa140a-b562-4541-a3ae-6fa50f309eee service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:14.320708 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:14.321346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:14.321609 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:14.321912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] PATH_INFO: `/v3/users/411c6c18366f487ab38980cf00e77af6` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:14.323078 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=411c6c18366f487ab38980cf00e77af6)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:14.324986 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:14.501768 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411c6c18366f487ab38980cf00e77af6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.510296 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:14.514573 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411c6c18366f487ab38980cf00e77af6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.521628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411c6c18366f487ab38980cf00e77af6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.522245 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411c6c18366f487ab38980cf00e77af6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.522850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-2d142f81-cf95-4d56-8897-97af1a85bd56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411c6c18366f487ab38980cf00e77af6'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:14.586012 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1394/2815] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:14 2022] DELETE /identity/v3/users/411c6c18366f487ab38980cf00e77af6 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:14.970694 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:14.975340 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:14.988220 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d447fe-32f2-4d35-8d8c-24bff02163e4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:15.043576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:15.049297 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:15.049928 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.050231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.050508 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] PATH_INFO: `/v3/projects/cfbd8b42779c48d79b991474f0352628` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.051644 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfbd8b42779c48d79b991474f0352628)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:15.053548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:15.136121 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfbd8b42779c48d79b991474f0352628', 'request_id': 'req-49513cbc-809c-4c5a-ad86-cdb2acbb5813'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:15.174298 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfbd8b42779c48d79b991474f0352628', 'request_id': 'req-49513cbc-809c-4c5a-ad86-cdb2acbb5813'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:15.179998 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfbd8b42779c48d79b991474f0352628', 'request_id': 'req-49513cbc-809c-4c5a-ad86-cdb2acbb5813'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:15.191483 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] The token cache is being invalidate because project cfbd8b42779c48d79b991474f0352628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:15.191773 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:15.192421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-49513cbc-809c-4c5a-ad86-cdb2acbb5813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:15.193915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1422/2816] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:14 2022] DELETE /identity/v3/projects/cfbd8b42779c48d79b991474f0352628 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:15.198747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a96a8-f3b5-4bf5-8ab0-3eaa5f13d118 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.199066 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a96a8-f3b5-4bf5-8ab0-3eaa5f13d118 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.199342 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a96a8-f3b5-4bf5-8ab0-3eaa5f13d118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.219115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c41a96a8-f3b5-4bf5-8ab0-3eaa5f13d118 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:15.220322 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c41a96a8-f3b5-4bf5-8ab0-3eaa5f13d118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:15.281586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1395/2817] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:15.290071 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a327439-e41d-4338-8a08-0847651aeb5c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.290355 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a327439-e41d-4338-8a08-0847651aeb5c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.290729 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a327439-e41d-4338-8a08-0847651aeb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.310258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0a327439-e41d-4338-8a08-0847651aeb5c None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:15.311474 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0a327439-e41d-4338-8a08-0847651aeb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:15.320027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1423/2818] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:15.324227 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a96a8-f3b5-4bf5-8ab0-3eaa5f13d118 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:15.334252 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a96a8-f3b5-4bf5-8ab0-3eaa5f13d118 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:15.338747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:15.339371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.339647 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.339914 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.340577 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:36:15.340872 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:15.342451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:15.353557 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1396/2819] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:36:15.357891 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a327439-e41d-4338-8a08-0847651aeb5c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:15.367049 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a327439-e41d-4338-8a08-0847651aeb5c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:15.371585 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73a8e-fd76-455c-930f-0833daa40a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:15.372188 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73a8e-fd76-455c-930f-0833daa40a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.372482 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73a8e-fd76-455c-930f-0833daa40a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.372748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73a8e-fd76-455c-930f-0833daa40a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.373360 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c73a8e-fd76-455c-930f-0833daa40a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:15.375252 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c73a8e-fd76-455c-930f-0833daa40a55 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:15.474404 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1424/2820] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:36:15 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:36:15.478757 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:15.489538 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c05d3-7407-45df-ae6a-fdfa58b92721 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:15.544591 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1e7ce-4043-444c-b9e6-dd66dfcd0c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:15.545186 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1e7ce-4043-444c-b9e6-dd66dfcd0c0b admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.545464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1e7ce-4043-444c-b9e6-dd66dfcd0c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.545737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1e7ce-4043-444c-b9e6-dd66dfcd0c0b admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.546357 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1e7ce-4043-444c-b9e6-dd66dfcd0c0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:15.548291 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1e7ce-4043-444c-b9e6-dd66dfcd0c0b admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:15.661115 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1397/2821] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:36:15 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:36:15.665362 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73a8e-fd76-455c-930f-0833daa40a55 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:15.676557 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73a8e-fd76-455c-930f-0833daa40a55 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:15.681286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85052401-8201-4486-a5d2-a465d30960eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:15.681893 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85052401-8201-4486-a5d2-a465d30960eb admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.682153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85052401-8201-4486-a5d2-a465d30960eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.682427 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85052401-8201-4486-a5d2-a465d30960eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.683103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85052401-8201-4486-a5d2-a465d30960eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:15.684674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85052401-8201-4486-a5d2-a465d30960eb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:15.694685 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1425/2822] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:15.698743 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1e7ce-4043-444c-b9e6-dd66dfcd0c0b admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:15.709280 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1e7ce-4043-444c-b9e6-dd66dfcd0c0b admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:15.713865 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8b9a1-7080-4050-9e83-9e34f1ad7a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:15.714456 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8b9a1-7080-4050-9e83-9e34f1ad7a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.714740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8b9a1-7080-4050-9e83-9e34f1ad7a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.715031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8b9a1-7080-4050-9e83-9e34f1ad7a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.715707 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8b9a1-7080-4050-9e83-9e34f1ad7a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:15.717300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8b9a1-7080-4050-9e83-9e34f1ad7a0f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:15.728362 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1398/2823] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:15.732645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85052401-8201-4486-a5d2-a465d30960eb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:15.742786 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85052401-8201-4486-a5d2-a465d30960eb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:15.747433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b8222-34ce-45a7-b713-7603e8021c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:15.752756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b8222-34ce-45a7-b713-7603e8021c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.752756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b8222-34ce-45a7-b713-7603e8021c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.752756 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b8222-34ce-45a7-b713-7603e8021c9d admin admin] PATH_INFO: `/v3/projects/eca283a03c78442eb7f94cb197bbb659/users/4b64202ecff64c4a8affd11aa53c6e8a/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.763450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b8222-34ce-45a7-b713-7603e8021c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca283a03c78442eb7f94cb197bbb659, user_id=4b64202ecff64c4a8affd11aa53c6e8a, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:15.766204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b8222-34ce-45a7-b713-7603e8021c9d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:15.923391 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1426/2824] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:15 2022] PUT /identity/v3/projects/eca283a03c78442eb7f94cb197bbb659/users/4b64202ecff64c4a8affd11aa53c6e8a/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:15.933778 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e28110-9410-42a5-ad92-b4b65aef5c86 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:15.934061 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e28110-9410-42a5-ad92-b4b65aef5c86 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:15.934334 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e28110-9410-42a5-ad92-b4b65aef5c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:15.961654 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-21e28110-9410-42a5-ad92-b4b65aef5c86 None None] MFA Rules not processed for user `4b64202ecff64c4a8affd11aa53c6e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:15.962862 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-21e28110-9410-42a5-ad92-b4b65aef5c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:15.993238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:16.028736 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1399/2825] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:16.033237 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b8222-34ce-45a7-b713-7603e8021c9d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:16.096527 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b8222-34ce-45a7-b713-7603e8021c9d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:16.101302 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148849-e36b-4b84-9cd8-2d0948e36121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:16.101903 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148849-e36b-4b84-9cd8-2d0948e36121 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.102184 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148849-e36b-4b84-9cd8-2d0948e36121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.102457 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148849-e36b-4b84-9cd8-2d0948e36121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.103114 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74148849-e36b-4b84-9cd8-2d0948e36121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:16.105022 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74148849-e36b-4b84-9cd8-2d0948e36121 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:16.182478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1427/2826] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:36:16 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:36:16.186610 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e28110-9410-42a5-ad92-b4b65aef5c86 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:16.197888 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e28110-9410-42a5-ad92-b4b65aef5c86 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:16.252033 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f482-8ca9-44d9-a400-bbf7e8c45243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:16.252627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f482-8ca9-44d9-a400-bbf7e8c45243 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.252908 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f482-8ca9-44d9-a400-bbf7e8c45243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.253176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f482-8ca9-44d9-a400-bbf7e8c45243 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.254346 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200f482-8ca9-44d9-a400-bbf7e8c45243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:16.256272 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200f482-8ca9-44d9-a400-bbf7e8c45243 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:16.357262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1400/2827] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:36:16 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:36:16.361198 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148849-e36b-4b84-9cd8-2d0948e36121 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:16.371602 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148849-e36b-4b84-9cd8-2d0948e36121 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:16.376304 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085355f-571f-43f5-8489-bffa0e269167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:16.376901 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085355f-571f-43f5-8489-bffa0e269167 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.377181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085355f-571f-43f5-8489-bffa0e269167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.377453 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085355f-571f-43f5-8489-bffa0e269167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.378092 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085355f-571f-43f5-8489-bffa0e269167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:16.380292 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085355f-571f-43f5-8489-bffa0e269167 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:16.382357 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:16.391107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1428/2828] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:16.395819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f482-8ca9-44d9-a400-bbf7e8c45243 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:16.405673 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f482-8ca9-44d9-a400-bbf7e8c45243 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:16.410295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c17bc-6be2-4538-9f12-0747918a8048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:16.410871 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c17bc-6be2-4538-9f12-0747918a8048 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.411266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c17bc-6be2-4538-9f12-0747918a8048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.411541 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c17bc-6be2-4538-9f12-0747918a8048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.412177 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c17bc-6be2-4538-9f12-0747918a8048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:16.413746 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c17bc-6be2-4538-9f12-0747918a8048 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:16.423528 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1401/2829] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:36:16.429566 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085355f-571f-43f5-8489-bffa0e269167 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:16.437856 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085355f-571f-43f5-8489-bffa0e269167 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:16.443170 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0199c-9fe3-443b-a038-964226d21d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:16.443815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0199c-9fe3-443b-a038-964226d21d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.444096 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0199c-9fe3-443b-a038-964226d21d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.444373 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0199c-9fe3-443b-a038-964226d21d28 admin admin] PATH_INFO: `/v3/projects/9e668530a8194368bfe42e3891bbded4/users/5f837dcd52b44935a337874002826195/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.459259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0199c-9fe3-443b-a038-964226d21d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e668530a8194368bfe42e3891bbded4, user_id=5f837dcd52b44935a337874002826195, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:16.461884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0199c-9fe3-443b-a038-964226d21d28 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:16.528630 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1429/2830] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:16 2022] PUT /identity/v3/projects/9e668530a8194368bfe42e3891bbded4/users/5f837dcd52b44935a337874002826195/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:16.540256 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e66fb1-c370-48c3-9291-1d2f8c081c1d None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.540568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e66fb1-c370-48c3-9291-1d2f8c081c1d None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.540841 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e66fb1-c370-48c3-9291-1d2f8c081c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.569117 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-35e66fb1-c370-48c3-9291-1d2f8c081c1d None None] MFA Rules not processed for user `5f837dcd52b44935a337874002826195`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:16.570310 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-35e66fb1-c370-48c3-9291-1d2f8c081c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:16.632172 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1402/2831] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:16.636657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0199c-9fe3-443b-a038-964226d21d28 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:16.698967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0199c-9fe3-443b-a038-964226d21d28 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:16.703559 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed533-4851-4acc-920b-d32c1f41caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:16.704156 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed533-4851-4acc-920b-d32c1f41caf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.704438 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed533-4851-4acc-920b-d32c1f41caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.704712 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed533-4851-4acc-920b-d32c1f41caf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.705325 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ed533-4851-4acc-920b-d32c1f41caf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:16.707614 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ed533-4851-4acc-920b-d32c1f41caf1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:16.787627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1430/2832] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:36:16 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:36:16.791756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e66fb1-c370-48c3-9291-1d2f8c081c1d None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:16.802424 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e66fb1-c370-48c3-9291-1d2f8c081c1d None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:16.840373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:16.858529 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdb12d-8024-469f-8e18-3a13955d21a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:16.859259 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdb12d-8024-469f-8e18-3a13955d21a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.859539 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdb12d-8024-469f-8e18-3a13955d21a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.859843 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdb12d-8024-469f-8e18-3a13955d21a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.860472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdb12d-8024-469f-8e18-3a13955d21a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:16.862430 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdb12d-8024-469f-8e18-3a13955d21a9 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:16.963672 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1403/2833] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:36:16 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:36:16.967759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed533-4851-4acc-920b-d32c1f41caf1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:16.978155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed533-4851-4acc-920b-d32c1f41caf1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:16.982837 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d8676-fa96-44a8-9abc-5c483d874605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:16.983449 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d8676-fa96-44a8-9abc-5c483d874605 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:16.983726 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d8676-fa96-44a8-9abc-5c483d874605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:16.983995 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d8676-fa96-44a8-9abc-5c483d874605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:16.984627 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d8676-fa96-44a8-9abc-5c483d874605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:16.986288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d8676-fa96-44a8-9abc-5c483d874605 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:16.996870 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1431/2834] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:17.000943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdb12d-8024-469f-8e18-3a13955d21a9 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:17.011629 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdb12d-8024-469f-8e18-3a13955d21a9 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:17.016192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472839-fe26-42b3-af78-7649d2e9d8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:17.017077 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472839-fe26-42b3-af78-7649d2e9d8f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:17.017355 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472839-fe26-42b3-af78-7649d2e9d8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:17.017625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472839-fe26-42b3-af78-7649d2e9d8f2 admin admin] PATH_INFO: `/v3/projects/f1649f66d971493082148e77b8f24654/users/1e33f72408164d01a6e2565399db7006/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:17.032917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98472839-fe26-42b3-af78-7649d2e9d8f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1649f66d971493082148e77b8f24654, user_id=1e33f72408164d01a6e2565399db7006, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:17.035756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98472839-fe26-42b3-af78-7649d2e9d8f2 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:17.106229 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1404/2835] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:16 2022] PUT /identity/v3/projects/f1649f66d971493082148e77b8f24654/users/1e33f72408164d01a6e2565399db7006/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:17.110254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d8676-fa96-44a8-9abc-5c483d874605 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:17.120569 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d8676-fa96-44a8-9abc-5c483d874605 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:17.177789 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca16c3d-fb65-4925-8397-e64e4e9a2fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:17.178385 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca16c3d-fb65-4925-8397-e64e4e9a2fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:17.178657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca16c3d-fb65-4925-8397-e64e4e9a2fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:17.178926 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca16c3d-fb65-4925-8397-e64e4e9a2fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:17.179588 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca16c3d-fb65-4925-8397-e64e4e9a2fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:17.181153 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca16c3d-fb65-4925-8397-e64e4e9a2fc8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:17.190728 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1432/2836] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:17.195079 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472839-fe26-42b3-af78-7649d2e9d8f2 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:17.205093 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472839-fe26-42b3-af78-7649d2e9d8f2 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:17.210247 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:17.210885 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:17.211192 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:17.211464 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] PATH_INFO: `/v3/projects/f1649f66d971493082148e77b8f24654/users/1e33f72408164d01a6e2565399db7006/roles/f72d3388f13149b483355331380f040b` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:17.213373 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1649f66d971493082148e77b8f24654, user_id=1e33f72408164d01a6e2565399db7006, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:17.216059 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:17.226237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1405/2837] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:17 2022] PUT /identity/v3/projects/f1649f66d971493082148e77b8f24654/users/1e33f72408164d01a6e2565399db7006/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:17.236182 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67c98b-fa48-4ec0-853c-74cf49a55eac None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:17.236469 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67c98b-fa48-4ec0-853c-74cf49a55eac None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:17.236746 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67c98b-fa48-4ec0-853c-74cf49a55eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:17.264513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8f67c98b-fa48-4ec0-853c-74cf49a55eac None None] MFA Rules not processed for user `1e33f72408164d01a6e2565399db7006`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:17.265705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8f67c98b-fa48-4ec0-853c-74cf49a55eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:17.297203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:17.331815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1433/2838] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 12:36:17.342262 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:17.349568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:17.400289 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ea92a-ce51-463a-b390-5572a1507db7 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:17.403850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcfe00-6d23-4134-bf1b-d42feb375411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:17.404437 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcfe00-6d23-4134-bf1b-d42feb375411 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:17.404713 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcfe00-6d23-4134-bf1b-d42feb375411 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:17.404981 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcfe00-6d23-4134-bf1b-d42feb375411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:17.413806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcfe00-6d23-4134-bf1b-d42feb375411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:17.415254 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcfe00-6d23-4134-bf1b-d42feb375411 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:17.424297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1406/2839] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:18.482009 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67c98b-fa48-4ec0-853c-74cf49a55eac None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:18.483421 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8f67c98b-fa48-4ec0-853c-74cf49a55eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:18.495554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67c98b-fa48-4ec0-853c-74cf49a55eac None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:18.499181 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e23c-1961-47ef-b94d-87b933cd81bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:18.499836 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e23c-1961-47ef-b94d-87b933cd81bf service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:18.500123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e23c-1961-47ef-b94d-87b933cd81bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:18.500398 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519e23c-1961-47ef-b94d-87b933cd81bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:18.557778 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519e23c-1961-47ef-b94d-87b933cd81bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:18.559234 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519e23c-1961-47ef-b94d-87b933cd81bf service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:18.568719 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1434/2840] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:23.389747 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcfe00-6d23-4134-bf1b-d42feb375411 service cinder] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:23.399391 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcfe00-6d23-4134-bf1b-d42feb375411 service cinder] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:23.405040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e99de-4725-4c16-a887-116d4b0f5a05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:23.405040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e99de-4725-4c16-a887-116d4b0f5a05 service cinder] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:23.405040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e99de-4725-4c16-a887-116d4b0f5a05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:23.405040 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e99de-4725-4c16-a887-116d4b0f5a05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:23.460200 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e99de-4725-4c16-a887-116d4b0f5a05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:23.461684 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e99de-4725-4c16-a887-116d4b0f5a05 service cinder] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:23.470612 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1407/2841] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:27.216488 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e23c-1961-47ef-b94d-87b933cd81bf service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:27.281045 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519e23c-1961-47ef-b94d-87b933cd81bf service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:27.285865 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:27.286492 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:27.286777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:27.287089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] PATH_INFO: `/v3/users/4b64202ecff64c4a8affd11aa53c6e8a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:27.288244 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b64202ecff64c4a8affd11aa53c6e8a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:27.290141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:27.485131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b64202ecff64c4a8affd11aa53c6e8a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:27.492041 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b64202ecff64c4a8affd11aa53c6e8a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:27.501991 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:27.504839 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b64202ecff64c4a8affd11aa53c6e8a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:27.505512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b64202ecff64c4a8affd11aa53c6e8a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:27.506101 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b64202ecff64c4a8affd11aa53c6e8a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:27.579907 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1435/2842] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:27 2022] DELETE /identity/v3/users/4b64202ecff64c4a8affd11aa53c6e8a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:27.585122 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b259c-5f62-46a4-bc87-8c119a97e2fd None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:27.585635 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b259c-5f62-46a4-bc87-8c119a97e2fd None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:27.585741 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b259c-5f62-46a4-bc87-8c119a97e2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:27.604244 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-c54b259c-5f62-46a4-bc87-8c119a97e2fd None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:27.605422 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c54b259c-5f62-46a4-bc87-8c119a97e2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:27.635352 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:27.664912 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1408/2843] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:27.674673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:27.676612 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:27.733027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434c8b-82eb-436f-9d83-37d2c4c11242 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:27.736695 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7cdac-290f-414f-a32f-8974099b76c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:27.737229 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cdac-290f-414f-a32f-8974099b76c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:27.737507 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cdac-290f-414f-a32f-8974099b76c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:27.737777 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cdac-290f-414f-a32f-8974099b76c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:27.747027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7cdac-290f-414f-a32f-8974099b76c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:27.748450 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7cdac-290f-414f-a32f-8974099b76c2 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:27.758376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1436/2844] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:28.147756 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b259c-5f62-46a4-bc87-8c119a97e2fd None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:28.149805 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c54b259c-5f62-46a4-bc87-8c119a97e2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:28.162699 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b259c-5f62-46a4-bc87-8c119a97e2fd None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:28.167889 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:28.168525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:28.168802 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:28.169082 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] PATH_INFO: `/v3/projects/eca283a03c78442eb7f94cb197bbb659` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:28.170204 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca283a03c78442eb7f94cb197bbb659)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:28.172303 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:28.295032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca283a03c78442eb7f94cb197bbb659', 'request_id': 'req-bc96c38f-f04d-4662-a5dd-2d066dbf110c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.357191 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca283a03c78442eb7f94cb197bbb659', 'request_id': 'req-bc96c38f-f04d-4662-a5dd-2d066dbf110c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.362975 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca283a03c78442eb7f94cb197bbb659', 'request_id': 'req-bc96c38f-f04d-4662-a5dd-2d066dbf110c'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.374649 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] The token cache is being invalidate because project eca283a03c78442eb7f94cb197bbb659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:28.374955 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.375604 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.377151 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1409/2845] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:28 2022] DELETE /identity/v3/projects/eca283a03c78442eb7f94cb197bbb659 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:28.381328 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7cdac-290f-414f-a32f-8974099b76c2 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:28.390967 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7cdac-290f-414f-a32f-8974099b76c2 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:28.447513 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:28.448135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:28.448416 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:28.448689 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] PATH_INFO: `/v3/users/5f837dcd52b44935a337874002826195` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:28.449809 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f837dcd52b44935a337874002826195)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:28.451741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:28.621495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f837dcd52b44935a337874002826195'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.628172 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f837dcd52b44935a337874002826195'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.639955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f837dcd52b44935a337874002826195'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.640626 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f837dcd52b44935a337874002826195'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.641225 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f837dcd52b44935a337874002826195'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:28.699674 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1437/2846] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:28 2022] DELETE /identity/v3/users/5f837dcd52b44935a337874002826195 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:29.073385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:29.074910 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:29.089533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96c38f-f04d-4662-a5dd-2d066dbf110c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:29.147674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:29.148217 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:29.148494 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:29.148762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] PATH_INFO: `/v3/projects/9e668530a8194368bfe42e3891bbded4` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:29.149864 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e668530a8194368bfe42e3891bbded4)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:29.151777 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:29.248198 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e668530a8194368bfe42e3891bbded4', 'request_id': 'req-796e0247-2fb5-4e75-bd4e-db148fb55d72'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.311880 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e668530a8194368bfe42e3891bbded4', 'request_id': 'req-796e0247-2fb5-4e75-bd4e-db148fb55d72'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.314060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:29.318867 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e668530a8194368bfe42e3891bbded4', 'request_id': 'req-796e0247-2fb5-4e75-bd4e-db148fb55d72'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.330154 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] The token cache is being invalidate because project 9e668530a8194368bfe42e3891bbded4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:29.330442 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.331097 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.332828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1410/2847] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:29 2022] DELETE /identity/v3/projects/9e668530a8194368bfe42e3891bbded4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:29.336815 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:29.338219 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:29.350605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37b3f7-05c5-4d37-ad0f-b2009e5c1cff admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:29.355179 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:29.407982 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:29.408608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:29.408808 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:29.409080 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] PATH_INFO: `/v3/users/1e33f72408164d01a6e2565399db7006` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:29.410288 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e33f72408164d01a6e2565399db7006)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:29.412393 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:29.594617 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e33f72408164d01a6e2565399db7006'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.601554 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e33f72408164d01a6e2565399db7006'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.613678 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e33f72408164d01a6e2565399db7006'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.614537 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e33f72408164d01a6e2565399db7006'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.615273 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7c335744-c506-49d2-a9d0-0387aadd7d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e33f72408164d01a6e2565399db7006'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:29.711192 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1438/2848] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:29 2022] DELETE /identity/v3/users/1e33f72408164d01a6e2565399db7006 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:30.079210 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:30.080671 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:30.093762 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e0247-2fb5-4e75-bd4e-db148fb55d72 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:30.158576 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:30.159237 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.159547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.159824 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] PATH_INFO: `/v3/projects/f1649f66d971493082148e77b8f24654` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.161088 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1649f66d971493082148e77b8f24654)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:30.163026 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:30.248785 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1649f66d971493082148e77b8f24654', 'request_id': 'req-93d2e9f1-cc35-4033-bc80-31b00aab977e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:30.288752 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1649f66d971493082148e77b8f24654', 'request_id': 'req-93d2e9f1-cc35-4033-bc80-31b00aab977e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:30.294409 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1649f66d971493082148e77b8f24654', 'request_id': 'req-93d2e9f1-cc35-4033-bc80-31b00aab977e'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:30.306718 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] The token cache is being invalidate because project f1649f66d971493082148e77b8f24654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:30.307157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:30.307840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-93d2e9f1-cc35-4033-bc80-31b00aab977e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:30.309423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1411/2849] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:30 2022] DELETE /identity/v3/projects/f1649f66d971493082148e77b8f24654 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:30.315409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb91b-9167-402e-a486-fe00f4acf573 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.315722 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb91b-9167-402e-a486-fe00f4acf573 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.315996 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb91b-9167-402e-a486-fe00f4acf573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.335326 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d9ecb91b-9167-402e-a486-fe00f4acf573 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:30.336514 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d9ecb91b-9167-402e-a486-fe00f4acf573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:30.394451 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1439/2850] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:30.403924 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c49b-beca-4d84-91a8-d67237fcbfed None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.404243 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c49b-beca-4d84-91a8-d67237fcbfed None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.404522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c49b-beca-4d84-91a8-d67237fcbfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.424290 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-b659c49b-beca-4d84-91a8-d67237fcbfed None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:30.425487 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-b659c49b-beca-4d84-91a8-d67237fcbfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:30.434049 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1412/2851] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:30.438089 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecb91b-9167-402e-a486-fe00f4acf573 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:30.448005 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecb91b-9167-402e-a486-fe00f4acf573 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:30.452544 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:30.453131 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.453411 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.453682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.454343 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:36:30.454645 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:30.456258 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:30.467796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1440/2852] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:36:30.471874 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c49b-beca-4d84-91a8-d67237fcbfed None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:30.481451 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c49b-beca-4d84-91a8-d67237fcbfed None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:30.486053 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eebd5-dea4-4e51-af14-7d3bec608c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:30.486637 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eebd5-dea4-4e51-af14-7d3bec608c7c admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.486917 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eebd5-dea4-4e51-af14-7d3bec608c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.487216 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eebd5-dea4-4e51-af14-7d3bec608c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.487840 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96eebd5-dea4-4e51-af14-7d3bec608c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:30.490340 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96eebd5-dea4-4e51-af14-7d3bec608c7c admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:30.627732 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1413/2853] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:36:30 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 12:36:30.631975 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:30.642299 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7944ec0-cb9d-43bb-8bee-c273c5cfca4e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:30.654207 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:30.699066 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c8db4-9d04-47ca-9174-24b852c573e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:30.699682 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c8db4-9d04-47ca-9174-24b852c573e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.700155 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c8db4-9d04-47ca-9174-24b852c573e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.700790 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c8db4-9d04-47ca-9174-24b852c573e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.701536 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c8db4-9d04-47ca-9174-24b852c573e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:30.703571 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c8db4-9d04-47ca-9174-24b852c573e0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:30.791568 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1441/2854] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:36:30 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:36:30.795836 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eebd5-dea4-4e51-af14-7d3bec608c7c admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:30.806381 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eebd5-dea4-4e51-af14-7d3bec608c7c admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:30.811017 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d5820-1a5c-4230-ab85-8c4195df24b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:30.811626 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d5820-1a5c-4230-ab85-8c4195df24b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.811904 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d5820-1a5c-4230-ab85-8c4195df24b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.812178 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d5820-1a5c-4230-ab85-8c4195df24b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.812812 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d5820-1a5c-4230-ab85-8c4195df24b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:30.814423 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d5820-1a5c-4230-ab85-8c4195df24b4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:30.824522 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1414/2855] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:30.828254 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c8db4-9d04-47ca-9174-24b852c573e0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:30.838869 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c8db4-9d04-47ca-9174-24b852c573e0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:30.843489 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b07cf0-a040-42f2-8cdb-40d226926b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:30.844073 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07cf0-a040-42f2-8cdb-40d226926b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.846027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07cf0-a040-42f2-8cdb-40d226926b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.846027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07cf0-a040-42f2-8cdb-40d226926b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.846027 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b07cf0-a040-42f2-8cdb-40d226926b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:30.846874 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b07cf0-a040-42f2-8cdb-40d226926b7a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:30.858382 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1442/2856] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:30.862476 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d5820-1a5c-4230-ab85-8c4195df24b4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:30.872688 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d5820-1a5c-4230-ab85-8c4195df24b4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:30.877319 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad4e73-4273-47bd-82ff-2eb821427e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:30.878007 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad4e73-4273-47bd-82ff-2eb821427e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.878238 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad4e73-4273-47bd-82ff-2eb821427e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.878511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad4e73-4273-47bd-82ff-2eb821427e53 admin admin] PATH_INFO: `/v3/projects/bfda8ff3d2bd4c5488fa5cf29b630713/users/d0e09166a2834d3d901bc43d607e97d0/roles/347174cd91d94848aec704f00988e871` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:30.893083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad4e73-4273-47bd-82ff-2eb821427e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfda8ff3d2bd4c5488fa5cf29b630713, user_id=d0e09166a2834d3d901bc43d607e97d0, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:30.896432 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad4e73-4273-47bd-82ff-2eb821427e53 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:30.967392 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1415/2857] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:30 2022] PUT /identity/v3/projects/bfda8ff3d2bd4c5488fa5cf29b630713/users/d0e09166a2834d3d901bc43d607e97d0/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:30.979176 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623a42-f522-48f9-bba1-3baece04c465 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:30.979465 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623a42-f522-48f9-bba1-3baece04c465 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:30.979741 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623a42-f522-48f9-bba1-3baece04c465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.008705 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b9623a42-f522-48f9-bba1-3baece04c465 None None] MFA Rules not processed for user `d0e09166a2834d3d901bc43d607e97d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:31.009948 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b9623a42-f522-48f9-bba1-3baece04c465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:31.072936 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1443/2858] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:31.077583 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad4e73-4273-47bd-82ff-2eb821427e53 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:31.091295 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:31.139060 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad4e73-4273-47bd-82ff-2eb821427e53 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:31.143881 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4dc25-e221-41c4-8d6c-f247c2ce0518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:31.145090 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4dc25-e221-41c4-8d6c-f247c2ce0518 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:31.145460 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4dc25-e221-41c4-8d6c-f247c2ce0518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:31.145737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4dc25-e221-41c4-8d6c-f247c2ce0518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.146350 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4dc25-e221-41c4-8d6c-f247c2ce0518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:31.148278 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4dc25-e221-41c4-8d6c-f247c2ce0518 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:31.227096 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1416/2859] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:36:31 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:36:31.231223 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9623a42-f522-48f9-bba1-3baece04c465 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:31.241957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9623a42-f522-48f9-bba1-3baece04c465 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:31.298711 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d14357-e613-4496-9b85-4767bfb59cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:31.299361 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14357-e613-4496-9b85-4767bfb59cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:31.299643 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14357-e613-4496-9b85-4767bfb59cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:31.299914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14357-e613-4496-9b85-4767bfb59cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.300604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d14357-e613-4496-9b85-4767bfb59cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:31.302495 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d14357-e613-4496-9b85-4767bfb59cb5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:31.380868 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1444/2860] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:36:31 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:36:31.384902 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4dc25-e221-41c4-8d6c-f247c2ce0518 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:31.395655 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4dc25-e221-41c4-8d6c-f247c2ce0518 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:31.400130 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb675668-e57b-4b0e-9c6f-eb943c0a66e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:31.400714 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb675668-e57b-4b0e-9c6f-eb943c0a66e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:31.400989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb675668-e57b-4b0e-9c6f-eb943c0a66e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:31.401261 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb675668-e57b-4b0e-9c6f-eb943c0a66e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.401927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb675668-e57b-4b0e-9c6f-eb943c0a66e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:31.403566 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb675668-e57b-4b0e-9c6f-eb943c0a66e7 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:31.414848 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1417/2861] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:36:31.418952 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d14357-e613-4496-9b85-4767bfb59cb5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:31.429370 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d14357-e613-4496-9b85-4767bfb59cb5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:31.433902 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d2e26-f0e2-4dad-a957-5e78733dfecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:31.434540 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d2e26-f0e2-4dad-a957-5e78733dfecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:31.434817 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d2e26-f0e2-4dad-a957-5e78733dfecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:31.435122 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d2e26-f0e2-4dad-a957-5e78733dfecf admin admin] PATH_INFO: `/v3/projects/9dcf32f76c6041ac81b8e70ad0d12199/users/f2f0739f4ad340abbe2826272e1677ce/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.446957 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:31.450784 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d2e26-f0e2-4dad-a957-5e78733dfecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dcf32f76c6041ac81b8e70ad0d12199, user_id=f2f0739f4ad340abbe2826272e1677ce, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:31.453512 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d2e26-f0e2-4dad-a957-5e78733dfecf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:31.524673 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1445/2862] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:31 2022] PUT /identity/v3/projects/9dcf32f76c6041ac81b8e70ad0d12199/users/f2f0739f4ad340abbe2826272e1677ce/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:31.528586 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb675668-e57b-4b0e-9c6f-eb943c0a66e7 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:31.539218 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb675668-e57b-4b0e-9c6f-eb943c0a66e7 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:31.593822 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459acb1-d3f4-44f2-8569-6df263769f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:31.594433 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459acb1-d3f4-44f2-8569-6df263769f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:31.594689 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459acb1-d3f4-44f2-8569-6df263769f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:31.594956 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459acb1-d3f4-44f2-8569-6df263769f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.595622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459acb1-d3f4-44f2-8569-6df263769f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:31.597230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459acb1-d3f4-44f2-8569-6df263769f41 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:31.607114 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1418/2863] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:31.611277 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d2e26-f0e2-4dad-a957-5e78733dfecf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:31.622473 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d2e26-f0e2-4dad-a957-5e78733dfecf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:31.627077 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:31.627725 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:31.628000 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:31.628268 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] PATH_INFO: `/v3/projects/9dcf32f76c6041ac81b8e70ad0d12199/users/f2f0739f4ad340abbe2826272e1677ce/roles/f72d3388f13149b483355331380f040b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.630285 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dcf32f76c6041ac81b8e70ad0d12199, user_id=f2f0739f4ad340abbe2826272e1677ce, role_id=f72d3388f13149b483355331380f040b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:31.632915 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:31.643477 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1446/2864] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:31 2022] PUT /identity/v3/projects/9dcf32f76c6041ac81b8e70ad0d12199/users/f2f0739f4ad340abbe2826272e1677ce/roles/f72d3388f13149b483355331380f040b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:36:31.659246 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78b77-6e65-4e4b-be99-1003aefbc7c0 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:31.659459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78b77-6e65-4e4b-be99-1003aefbc7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:31.659725 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78b77-6e65-4e4b-be99-1003aefbc7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.686754 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-3dc78b77-6e65-4e4b-be99-1003aefbc7c0 None None] MFA Rules not processed for user `f2f0739f4ad340abbe2826272e1677ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:31.688062 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3dc78b77-6e65-4e4b-be99-1003aefbc7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:31.753536 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1419/2865] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:31.764905 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:31.766847 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:31.825531 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101fde0-eaa5-40f3-84ef-f4aab9acf2ef admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:31.829260 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8fdef-a082-40a7-9e50-afe15ab96a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:31.829854 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8fdef-a082-40a7-9e50-afe15ab96a3f service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:31.830130 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8fdef-a082-40a7-9e50-afe15ab96a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:31.830487 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8fdef-a082-40a7-9e50-afe15ab96a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:31.839608 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8fdef-a082-40a7-9e50-afe15ab96a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:31.841018 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8fdef-a082-40a7-9e50-afe15ab96a3f service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:31.851367 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1447/2866] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:32.942744 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc78b77-6e65-4e4b-be99-1003aefbc7c0 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:32.952336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:33.004518 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc78b77-6e65-4e4b-be99-1003aefbc7c0 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:33.009072 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:33.011083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:33.011083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:33.011083 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] PATH_INFO: `/v3/users/d0e09166a2834d3d901bc43d607e97d0` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:33.011408 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e09166a2834d3d901bc43d607e97d0)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:33.013317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:33.244627 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e09166a2834d3d901bc43d607e97d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:33.256789 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e09166a2834d3d901bc43d607e97d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:33.263390 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e09166a2834d3d901bc43d607e97d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:33.264046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e09166a2834d3d901bc43d607e97d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:33.264640 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e09166a2834d3d901bc43d607e97d0'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:33.327650 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1420/2867] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:32 2022] DELETE /identity/v3/users/d0e09166a2834d3d901bc43d607e97d0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:33.332810 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc94189-00ed-4c7d-b2e3-73ad51751d04 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:33.333339 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc94189-00ed-4c7d-b2e3-73ad51751d04 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:33.333511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc94189-00ed-4c7d-b2e3-73ad51751d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:33.351677 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0cc94189-00ed-4c7d-b2e3-73ad51751d04 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:33.359440 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0cc94189-00ed-4c7d-b2e3-73ad51751d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:33.411078 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1448/2868] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:33.420593 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:33.421989 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:33.478887 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e206deb-18a7-4620-b2a3-8442fdb48cb5 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:33.482468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417d0f1-45eb-44b3-b3f8-05562d18c20b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:33.483102 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417d0f1-45eb-44b3-b3f8-05562d18c20b service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:33.483382 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417d0f1-45eb-44b3-b3f8-05562d18c20b service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:33.483652 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417d0f1-45eb-44b3-b3f8-05562d18c20b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:33.492622 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417d0f1-45eb-44b3-b3f8-05562d18c20b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:33.494042 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417d0f1-45eb-44b3-b3f8-05562d18c20b service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:33.503317 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1421/2869] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:33.914255 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc94189-00ed-4c7d-b2e3-73ad51751d04 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:33.915706 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0cc94189-00ed-4c7d-b2e3-73ad51751d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:33.928346 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc94189-00ed-4c7d-b2e3-73ad51751d04 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:33.933250 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:33.934199 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:33.936484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:33.936484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] PATH_INFO: `/v3/projects/bfda8ff3d2bd4c5488fa5cf29b630713` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:33.936484 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfda8ff3d2bd4c5488fa5cf29b630713)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:33.938333 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:34.033081 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfda8ff3d2bd4c5488fa5cf29b630713', 'request_id': 'req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.139519 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfda8ff3d2bd4c5488fa5cf29b630713', 'request_id': 'req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.146231 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfda8ff3d2bd4c5488fa5cf29b630713', 'request_id': 'req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.157354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] The token cache is being invalidate because project bfda8ff3d2bd4c5488fa5cf29b630713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:34.157642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.158291 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.159803 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1449/2870] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:33 2022] DELETE /identity/v3/projects/bfda8ff3d2bd4c5488fa5cf29b630713 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:34.164489 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417d0f1-45eb-44b3-b3f8-05562d18c20b service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:34.174155 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417d0f1-45eb-44b3-b3f8-05562d18c20b service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:34.203969 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:34.231372 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:34.231987 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:34.232266 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:34.232533 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] PATH_INFO: `/v3/users/f2f0739f4ad340abbe2826272e1677ce` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:34.233628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f0739f4ad340abbe2826272e1677ce)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:34.235666 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:34.399706 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f0739f4ad340abbe2826272e1677ce'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.412233 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f0739f4ad340abbe2826272e1677ce'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.419265 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f0739f4ad340abbe2826272e1677ce'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.421183 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f0739f4ad340abbe2826272e1677ce'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.421798 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-aa5ac7ec-146e-4f18-bc13-b91cea36f49f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f0739f4ad340abbe2826272e1677ce'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:34.488511 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1422/2871] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:36:34 2022] DELETE /identity/v3/users/f2f0739f4ad340abbe2826272e1677ce => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:34.889261 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:34.890698 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:34.899727 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:34.905210 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffdaa7-f88b-4ab0-9d02-819f6487b07c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:34.963734 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:34.964410 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:34.965002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:34.965002 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] PATH_INFO: `/v3/projects/9dcf32f76c6041ac81b8e70ad0d12199` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:34.966058 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dcf32f76c6041ac81b8e70ad0d12199)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:34.968037 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:35.111941 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dcf32f76c6041ac81b8e70ad0d12199', 'request_id': 'req-10893539-faf9-4197-8d55-ee5803bc24b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:35.197522 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dcf32f76c6041ac81b8e70ad0d12199', 'request_id': 'req-10893539-faf9-4197-8d55-ee5803bc24b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:35.203862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dcf32f76c6041ac81b8e70ad0d12199', 'request_id': 'req-10893539-faf9-4197-8d55-ee5803bc24b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:35.215914 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] The token cache is being invalidate because project 9dcf32f76c6041ac81b8e70ad0d12199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:36:35.216232 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:35.216866 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-10893539-faf9-4197-8d55-ee5803bc24b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:36:35.218352 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1450/2872] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:36:34 2022] DELETE /identity/v3/projects/9dcf32f76c6041ac81b8e70ad0d12199 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:36:35.223358 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e347-533b-4da0-b4d4-3bb945088de9 None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.223891 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e347-533b-4da0-b4d4-3bb945088de9 None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.224203 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e347-533b-4da0-b4d4-3bb945088de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.243988 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-a0c1e347-533b-4da0-b4d4-3bb945088de9 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:35.245128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-a0c1e347-533b-4da0-b4d4-3bb945088de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:35.304595 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1423/2873] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:35.313500 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492147ae-fb3d-445c-930e-ef9dd6812f43 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.314266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492147ae-fb3d-445c-930e-ef9dd6812f43 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.314266 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492147ae-fb3d-445c-930e-ef9dd6812f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.334396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-492147ae-fb3d-445c-930e-ef9dd6812f43 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:35.335638 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-492147ae-fb3d-445c-930e-ef9dd6812f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:35.343530 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1451/2874] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:35.347782 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1e347-533b-4da0-b4d4-3bb945088de9 None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:35.358347 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1e347-533b-4da0-b4d4-3bb945088de9 None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:35.362985 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:35.363547 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.363827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.364849 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] PATH_INFO: `/v3/domains` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.365505 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 12:36:35.365806 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:35.367436 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:35.378734 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1424/2875] 10.222.0.30 () {58 vars in 1153 bytes} [Sun Aug 7 12:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 12:36:35.383966 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492147ae-fb3d-445c-930e-ef9dd6812f43 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:35.393609 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492147ae-fb3d-445c-930e-ef9dd6812f43 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:35.398412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e837e-12e4-480e-9c26-691aea8c4348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:35.400224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e837e-12e4-480e-9c26-691aea8c4348 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.400224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e837e-12e4-480e-9c26-691aea8c4348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.400224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e837e-12e4-480e-9c26-691aea8c4348 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.400224 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e837e-12e4-480e-9c26-691aea8c4348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:35.402230 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e837e-12e4-480e-9c26-691aea8c4348 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:35.506792 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1452/2876] 10.222.0.30 () {60 vars in 1153 bytes} [Sun Aug 7 12:36:35 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:36:35.510948 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:35.521309 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bc0cb-3e85-43e4-ac01-d71cb483291e admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:35.553996 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:35.577207 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541f6c5-533e-4722-8b66-bc44168e2766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:35.577819 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541f6c5-533e-4722-8b66-bc44168e2766 admin admin] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.578140 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541f6c5-533e-4722-8b66-bc44168e2766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.578410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541f6c5-533e-4722-8b66-bc44168e2766 admin admin] PATH_INFO: `/v3/users` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.579046 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541f6c5-533e-4722-8b66-bc44168e2766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:35.580939 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541f6c5-533e-4722-8b66-bc44168e2766 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:35.677302 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1425/2877] 10.222.0.30 () {60 vars in 1144 bytes} [Sun Aug 7 12:36:35 2022] POST /identity/v3/users => generated 282 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 12:36:35.681433 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e837e-12e4-480e-9c26-691aea8c4348 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:35.691850 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e837e-12e4-480e-9c26-691aea8c4348 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:35.697259 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1921c-a946-4aa3-a837-f9c1c36f9bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:35.697858 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1921c-a946-4aa3-a837-f9c1c36f9bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.698125 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1921c-a946-4aa3-a837-f9c1c36f9bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.698396 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1921c-a946-4aa3-a837-f9c1c36f9bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.699203 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1921c-a946-4aa3-a837-f9c1c36f9bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:35.700795 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1921c-a946-4aa3-a837-f9c1c36f9bcd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:35.711471 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1453/2878] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 12:36:35.715426 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541f6c5-533e-4722-8b66-bc44168e2766 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:35.725700 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541f6c5-533e-4722-8b66-bc44168e2766 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:35.730325 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748945a-8be7-444b-b2d6-840af93b88af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:35.730905 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748945a-8be7-444b-b2d6-840af93b88af admin admin] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.731213 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748945a-8be7-444b-b2d6-840af93b88af admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.731481 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748945a-8be7-444b-b2d6-840af93b88af admin admin] PATH_INFO: `/v3/roles` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.732110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748945a-8be7-444b-b2d6-840af93b88af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:35.733737 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748945a-8be7-444b-b2d6-840af93b88af admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:35.743906 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1426/2879] 10.222.0.30 () {58 vars in 1122 bytes} [Sun Aug 7 12:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 12:36:35.753611 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1921c-a946-4aa3-a837-f9c1c36f9bcd admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:35.763994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1921c-a946-4aa3-a837-f9c1c36f9bcd admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:35.768840 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:35.769485 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.769762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.770036 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] PATH_INFO: `/v3/projects/0b8086f6f7e048608df323e03d821f99/users/e202fc17a7244fe79724cd7122e81919/roles/347174cd91d94848aec704f00988e871` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.785100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8086f6f7e048608df323e03d821f99, user_id=e202fc17a7244fe79724cd7122e81919, role_id=347174cd91d94848aec704f00988e871)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:35.787771 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:35.876707 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1454/2880] 10.222.0.30 () {60 vars in 1483 bytes} [Sun Aug 7 12:36:35 2022] PUT /identity/v3/projects/0b8086f6f7e048608df323e03d821f99/users/e202fc17a7244fe79724cd7122e81919/roles/347174cd91d94848aec704f00988e871 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:36:35.887568 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b70e18-aa5c-4a6c-8412-72634a3bfc5e None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:35.895417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b70e18-aa5c-4a6c-8412-72634a3bfc5e None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:35.895417 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b70e18-aa5c-4a6c-8412-72634a3bfc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:35.915611 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-15b70e18-aa5c-4a6c-8412-72634a3bfc5e None None] MFA Rules not processed for user `e202fc17a7244fe79724cd7122e81919`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:35.916809 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-15b70e18-aa5c-4a6c-8412-72634a3bfc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:35.978845 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1427/2881] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:35.990623 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:35.992375 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:36.025748 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:36.058759 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90ebb9-b077-4ba9-9ae4-ac61463c2a94 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:36.062486 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0792940b93b4c1998be971f6143a514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:36.068751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:36.068751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:36.068751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:36.073476 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:36.074904 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:36.084056 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1455/2882] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:36:39.970495 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece445f-cf57-440b-ba5e-bebf6c4f911f None None] REQUEST_METHOD: `POST` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:39.971089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece445f-cf57-440b-ba5e-bebf6c4f911f None None] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:39.971089 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece445f-cf57-440b-ba5e-bebf6c4f911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:39.991468 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.auth.core [None req-1ece445f-cf57-440b-ba5e-bebf6c4f911f None None] MFA Rules not processed for user `6c52cd52d25a42e29c8f049c94c903aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:36:39.992680 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-1ece445f-cf57-440b-ba5e-bebf6c4f911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:40.048722 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1428/2883] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:36:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:36:46.117750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=13 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:50.042176 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:36:58.679376 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:36:58.680983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:58.762567 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9386a2-ada0-40b9-a86c-0ebec3512956 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:36:58.768242 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffab9d8-adc0-4c0d-835a-d8087d362c9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:36:58.768841 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffab9d8-adc0-4c0d-835a-d8087d362c9c service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:36:58.769117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffab9d8-adc0-4c0d-835a-d8087d362c9c service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:36:58.769387 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffab9d8-adc0-4c0d-835a-d8087d362c9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:36:58.771094 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bffab9d8-adc0-4c0d-835a-d8087d362c9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:36:58.782377 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffab9d8-adc0-4c0d-835a-d8087d362c9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:36:58.784169 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffab9d8-adc0-4c0d-835a-d8087d362c9c service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:36:58.795068 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1456/2884] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:36:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:37:08.789774 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=34 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:37:15.519507 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece445f-cf57-440b-ba5e-bebf6c4f911f None None] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:37:15.581388 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece445f-cf57-440b-ba5e-bebf6c4f911f None None] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:37:15.585909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:37:15.586544 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:37:15.586827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:37:15.587159 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] PATH_INFO: `/v3/users/e202fc17a7244fe79724cd7122e81919` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:37:15.588410 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e202fc17a7244fe79724cd7122e81919)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:37:15.590330 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:37:15.723492 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e202fc17a7244fe79724cd7122e81919'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:15.735936 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e202fc17a7244fe79724cd7122e81919'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:15.742618 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e202fc17a7244fe79724cd7122e81919'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:15.743300 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e202fc17a7244fe79724cd7122e81919'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:15.743938 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e202fc17a7244fe79724cd7122e81919'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:15.808016 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1429/2885] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:37:15 2022] DELETE /identity/v3/users/e202fc17a7244fe79724cd7122e81919 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:37:15.812518 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd02eb-7064-4809-b4b5-523b367d7041 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:37:15.813024 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd02eb-7064-4809-b4b5-523b367d7041 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:37:15.813138 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd02eb-7064-4809-b4b5-523b367d7041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:37:15.831420 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4edd02eb-7064-4809-b4b5-523b367d7041 None None] MFA Rules not processed for user `6e390f818f21433d9d55c5e4c3ba79e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 12:37:15.832625 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4edd02eb-7064-4809-b4b5-523b367d7041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:37:15.890358 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1457/2886] 10.222.0.30 () {58 vars in 958 bytes} [Sun Aug 7 12:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 12:37:15.900783 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:37:15.902287 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:37:15.959119 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29ea96-ac22-4fab-a4e0-d124f7007178 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:37:15.962740 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffddfb-acb4-4e72-ac90-616cb2c880f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:37:15.963368 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffddfb-acb4-4e72-ac90-616cb2c880f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:37:15.963687 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffddfb-acb4-4e72-ac90-616cb2c880f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:37:15.963963 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffddfb-acb4-4e72-ac90-616cb2c880f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:37:15.976031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffddfb-acb4-4e72-ac90-616cb2c880f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:37:15.977474 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffddfb-acb4-4e72-ac90-616cb2c880f3 service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:37:15.987068 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1430/2887] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:37:16.386994 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd02eb-7064-4809-b4b5-523b367d7041 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:37:16.388548 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4edd02eb-7064-4809-b4b5-523b367d7041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:37:16.400884 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd02eb-7064-4809-b4b5-523b367d7041 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:37:16.405575 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:37:16.406204 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:37:16.406490 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:37:16.406804 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] PATH_INFO: `/v3/projects/0b8086f6f7e048608df323e03d821f99` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:37:16.407954 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8086f6f7e048608df323e03d821f99)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:37:16.409862 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:37:16.535100 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8086f6f7e048608df323e03d821f99', 'request_id': 'req-02fa0e54-a803-4821-82dd-5706fd6608c7'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:16.590864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8086f6f7e048608df323e03d821f99', 'request_id': 'req-02fa0e54-a803-4821-82dd-5706fd6608c7'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:16.596789 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8086f6f7e048608df323e03d821f99', 'request_id': 'req-02fa0e54-a803-4821-82dd-5706fd6608c7'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:16.608963 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] The token cache is being invalidate because project 0b8086f6f7e048608df323e03d821f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:37:16.609332 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:16.609973 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:37:16.611497 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1458/2888] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:37:16 2022] DELETE /identity/v3/projects/0b8086f6f7e048608df323e03d821f99 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 12:37:25.982796 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:37:26.605853 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:38:06.298014 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffddfb-acb4-4e72-ac90-616cb2c880f3 service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:38:06.306466 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffddfb-acb4-4e72-ac90-616cb2c880f3 service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:38:06.353853 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:38:06.354572 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:38:06.354855 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:38:06.355230 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:38:06.356919 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:06.366110 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:38:06.367590 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:38:06.376098 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1431/2889] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:38:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:38:16.372472 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=26 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:38:43.206271 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:38:43.207970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:43.295908 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa0e54-a803-4821-82dd-5706fd6608c7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:38:43.301641 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:38:43.302245 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:38:43.302524 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:38:43.302794 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:38:43.304561 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:43.317661 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:38:43.319287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:38:43.330713 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1459/2890] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:38:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:38:43.564774 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:38:43.566164 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:43.578157 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470f38e-93fd-421e-bd91-e9cbca31dcae service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:38:43.582128 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:38:43.582733 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:38:43.583027 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:38:43.583313 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:38:43.585073 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:43.598930 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:38:43.600473 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:38:43.611800 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1432/2891] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:38:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:38:44.794955 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:38:44.796381 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:44.807511 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1beb6-fab6-481c-b1f6-224c1adf5fbd service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:38:44.810983 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:38:44.811586 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:38:44.811864 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:38:44.812135 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:38:44.813751 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:44.827189 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:38:44.828576 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:38:44.839515 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1460/2892] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:38:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:38:46.040827 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:38:46.042297 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:46.057641 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071ed1-6c25-48fe-8479-abf4c4ae2f9d service neutron] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:38:46.063893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb2075e9fff3459881c13f81456ab198', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:38:46.064490 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:38:46.064766 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:38:46.065036 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:38:46.066990 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:46.080927 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:38:46.082519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:38:46.093525 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1433/2893] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:38:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:38:54.834443 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=40 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:38:56.088814 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=12 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:38:58.615965 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:38:58.624764 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ce156-e600-4d6b-b706-2cecd9c1e337 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:38:58.629035 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7aa31ea778140189b041939e3f01aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:38:58.629629 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:38:58.629911 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:38:58.630185 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:38:58.631906 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:38:58.695642 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:38:58.697123 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:38:58.707458 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1461/2894] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:39:01.876450 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:39:01.878298 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:39:01.947915 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a4591-7dfb-4cdc-ba53-079b422d2803 service nova] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:39:01.952797 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:39:01.955336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:39:01.955336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:39:01.955336 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:39:01.956532 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:39:01.967625 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:39:01.969893 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:39:01.980826 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1434/2895] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:39:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 12:39:05.555762 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:39:05.567846 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6fa78-d302-4493-a643-856cb04cc98f service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:39:05.573117 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:39:05.573723 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:39:05.574010 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:39:05.574287 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:39:05.576152 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:39:05.589605 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:39:05.591151 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:39:05.602107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1462/2896] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:39:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:39:05.665004 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:39:05.666500 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:39:05.683315 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7093a65-6ff1-4637-aca0-859476750c33 service placement] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:39:05.685909 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:39:05.686552 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:39:05.686828 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:39:05.687137 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] PATH_INFO: `/v3/users/6c52cd52d25a42e29c8f049c94c903aa` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:39:05.688943 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c52cd52d25a42e29c8f049c94c903aa)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:39:05.690850 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:39:05.888425 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c52cd52d25a42e29c8f049c94c903aa'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:05.900772 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c52cd52d25a42e29c8f049c94c903aa'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:05.908084 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c52cd52d25a42e29c8f049c94c903aa'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:05.908775 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c52cd52d25a42e29c8f049c94c903aa'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:05.909379 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c52cd52d25a42e29c8f049c94c903aa'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:05.977582 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1435/2897] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:39:05 2022] DELETE /identity/v3/users/6c52cd52d25a42e29c8f049c94c903aa => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:39:06.185412 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:39:06.196985 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac354ae-56c8-4206-b00a-f2c08404be65 service placement] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:39:06.254177 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:39:06.254799 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:39:06.255107 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:39:06.255380 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] PATH_INFO: `/v3/projects/0e2aa26b8ec54387b7c19fc0a2b9438f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:39:06.256478 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2aa26b8ec54387b7c19fc0a2b9438f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:39:06.258354 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:39:06.443042 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2aa26b8ec54387b7c19fc0a2b9438f', 'request_id': 'req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.498067 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2aa26b8ec54387b7c19fc0a2b9438f', 'request_id': 'req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.504657 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2aa26b8ec54387b7c19fc0a2b9438f', 'request_id': 'req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.515819 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] The token cache is being invalidate because project 0e2aa26b8ec54387b7c19fc0a2b9438f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:39:06.516103 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.516757 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.518286 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1463/2898] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:39:06 2022] DELETE /identity/v3/projects/0e2aa26b8ec54387b7c19fc0a2b9438f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:39:06.522674 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:39:06.524136 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:39:06.537519 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74f84f-9fdc-4c8f-bbe8-fb8c60e371f4 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:39:06.592976 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:39:06.593628 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:39:06.594396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:39:06.594663 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] PATH_INFO: `/v3/users/7ce4d8dd8b4143ca90a9ecddaa8603b5` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:39:06.595834 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce4d8dd8b4143ca90a9ecddaa8603b5)` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:39:06.597750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:39:06.600021 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:39:06.723668 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce4d8dd8b4143ca90a9ecddaa8603b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.735983 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce4d8dd8b4143ca90a9ecddaa8603b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.742691 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce4d8dd8b4143ca90a9ecddaa8603b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.743385 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce4d8dd8b4143ca90a9ecddaa8603b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.743993 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.notifications [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce4d8dd8b4143ca90a9ecddaa8603b5'} {{(pid=80432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:06.789031 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1436/2899] 10.222.0.30 () {58 vars in 1224 bytes} [Sun Aug 7 12:39:06 2022] DELETE /identity/v3/users/7ce4d8dd8b4143ca90a9ecddaa8603b5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:39:07.037652 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:39:07.039141 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:39:07.052970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad8125-491f-49c0-ae3a-ae9ea775f4bb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:39:07.062750 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:39:07.112502 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e390f818f21433d9d55c5e4c3ba79e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc097a340cd405e932b3797d5864d76', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:39:07.113133 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:39:07.113409 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:39:07.113732 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] PATH_INFO: `/v3/projects/bcd2cae596b94d399c13b72db54e1406` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:39:07.114872 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcd2cae596b94d399c13b72db54e1406)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:39:07.116796 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:39:07.258604 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcd2cae596b94d399c13b72db54e1406', 'request_id': 'req-77646210-7876-4cc6-88e1-815fedb8b0b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:07.332368 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcd2cae596b94d399c13b72db54e1406', 'request_id': 'req-77646210-7876-4cc6-88e1-815fedb8b0b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:07.338701 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcd2cae596b94d399c13b72db54e1406', 'request_id': 'req-77646210-7876-4cc6-88e1-815fedb8b0b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:07.350861 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] The token cache is being invalidate because project bcd2cae596b94d399c13b72db54e1406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 12:39:07.351190 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:07.351863 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-77646210-7876-4cc6-88e1-815fedb8b0b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 12:39:07.354970 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1464/2900] 10.222.0.30 () {58 vars in 1233 bytes} [Sun Aug 7 12:39:07 2022] DELETE /identity/v3/projects/bcd2cae596b94d399c13b72db54e1406 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 12:39:16.785396 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:39:17.347939 dv-cinder-849846-8-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 12:41:16.616623 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] Authenticating user token {{(pid=80432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 12:41:16.618032 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:41:16.753459 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019c6d-e899-4bd0-a546-fb5944ade5e1 admin admin] Validating token access rules against request {{(pid=80432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 12:41:16.810484 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3c39b-bb60-4050-8c58-2a592c3b232a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ee3d70937740b8adcb16ab57dc3031', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa20498b52fb441c8734f1813776cf2e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 12:41:16.811108 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3c39b-bb60-4050-8c58-2a592c3b232a service placement] REQUEST_METHOD: `GET` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 12:41:16.811387 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3c39b-bb60-4050-8c58-2a592c3b232a service placement] SCRIPT_NAME: `/identity` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 12:41:16.811679 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3c39b-bb60-4050-8c58-2a592c3b232a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 12:41:16.813371 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.fernet_utils [None req-4bb3c39b-bb60-4050-8c58-2a592c3b232a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 12:41:16.824199 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3c39b-bb60-4050-8c58-2a592c3b232a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 12:41:16.825750 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3c39b-bb60-4050-8c58-2a592c3b232a service placement] RBAC: Authorization granted {{(pid=80432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 12:41:16.835972 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: [pid: 80432|app: 0|req: 1437/2901] 10.222.0.30 () {60 vars in 1365 bytes} [Sun Aug 7 12:41:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 12:41:26.831212 dv-cinder-849846-8-iscsi devstack@keystone.service[80432]: DEBUG dbcounter [-] [80432] Writing DB stats keystone:SELECT=89 {{(pid=80432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}